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ABSTRACT ■ " . ' , " , . ' 

• The project aescribed is a re'nenai, of earlier nor Ic' on 
inaiviaiialized' Instruction for Data access (IIDI) , a progaaii assigned 
i?ersit y»^o-a i a-as esa«in »^earch lag^a-.daAafea,s e w I t h^ t h ft - 



coipttter actipg as assistant, This qnarteriy report discusses the 
©▼aiuation of IIDA through user interaction. Two types of evaluation 
were pert orn«dsfor«ati7eeTaiuatieD*^ 

and previaing feedbaclt -leading, to further refineBenti and sanative ' 
evaluation, wbich was concernea with, the iipaet aad effectiveness, of ' 
feh*.~lIDA^^systea.-^Foraatlve-e.valnati^ 



observation of systea use by three grcupsi conputer science aajors,, 
• ' actual users, ana library, ana inforaation science graduate stuaentt 
„^„i__„and_.faculty. P.rografflaing err^^^^^ 

% ekaained through this teat of the systea. ' Sumaative evaluation jras , 
g ggyrtfa-wt-th rough ^ s ervat i mi u£ auaiuhwb petfora^a by tiio #r.onpy 



cf novice us«rs--new students at thfc school of library and . 
inforaation science, and engineering students taking' a technical ' 
writing eour ge-- Ev aluative questionnaires wfre also c oa plet ed .b y the 
two groups of novice users, anTThe' results are tfieiXy suallfrle^t 
Search topics, guesticnniires, and conrse requireients for the novice 
users are contained in appendices, CSS), 
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^^^^^^K^^fe^^iS^^ ^^^^^ |t 

ti^ r^f^M^^w ^restiKd In J578# ca^;i0 ^to fii^^ ^^^^ 



-iJl^^ S^^ir tt^^igh 



ayffastad to hlffif to© sisplB that 

smee^^iftil ^aarcii. The user ijp'fMe -to MMry owt his seardi as he wants* 

--------- J. . - ^ - ^ : ^".-^ - " -- ■ 



In this ^ksffc±sm# tta piograni tis#s a ebri sst.af diagni^tia x©utines and 
nlta to Mmi't©^ the aetl^ty ©f ftm «©r and to prow.de ^a^^ua Jcinds of 



fMdba^ or asBistanoa. In addition r the iis€r has aowss, to several 3djada 
..d£.ha3p^^uld.Ji^^^©3u^ste _ — 

fha 'fiiird etorase ^presmtr advailged >earc^ trailing in that,^ as in 
ejEeraiae one, toe user is totrodyoed tiy m^titi^ 



Bie additional oomimdf ijMrodusad here Inolfi^ ^udh things as variations i 



on SaiCf # ^d the shorthand notations for DiMfOfe cOTMnto- In addition 
•Qie ^er Is introduced to a njrtber of otter advaiced topiCT. v 

ERIC ' . ■ . . — . " ■ • . , - , . ~^ 





V tha €na of^Frt^^^ 





sad : to ■ cofliMinas bCToad: :the basU^:^ leSf 



fl^#t saooad ^ereisas bad ^ 
fc^ in-^&a^yktssi.^ Hitt ^^^^ saosnd ^^^i^i pMvl^Jig a qurieus -tor , 



fli^ b^i^^^^t^TO^ ©f*fcha piojtot staff "i3/'C©noeraed w^^ 



, ^ysta^. Eor. axasplef ^smber of yia diapi©ftie r\ias*^JLn©D£3?ora ted into 



ttLreBtotd^^wr^ val^s were atft"^y; tet^ 'ttblttary 



©holce* .HG*#ev#tr the t^e gC th© system has 'cabled ^aviai©h pf same ©f tile 
thrssh©ld val43ss te^ayds more appropriate levels " . . * * • ^ 



In'siJmative evaluation^ ooneam of r the behavioral group is with 
#^^4n'ttp whi^ the ©bjeetivei oi tile project* hcve been met. In the b©^ ©f 



"T^thls -rfe^rt :tte%e is'^a fliB^et^^^oft iof the inltial^^oraative evaimtions^^©^ 
" 'ducted with the system well as a report ©n tw^Stlidias ©©nduoted at Drexel 
whi oh 'provide w|.th inf©mation on both formative and suroiativa evaluation. 
In y^^epgrt \#hiori follaws^^^ is o ne# ^ere 4d-M be 'a ^soussion of the 4 " 



• ■ 



eoniuc^ and results of two field studies whidi Constitute Uie-sironative 



* evaluation o4^IIDA* 
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atra efaac of tke XIDA system^ ft i%p«^^ble ta asJc ta«& '»ajor 



kizids of i^erfciais. ^^e £l»t of tji^ge Is abpat m££m^m of XlfiA 
.4^e system bpnmtes only 4p itt dsslst#nt« teafiag with, tbls-^te^f 1 oyi^ 

Is asTOaed the usss^ has pr&nQWly a raa^ae^la asdtmt of 



te^^ing ia%IMiOG sear^ingr amA ^gmgmm tZWi oaly.tfaraugh tte aesutance 
r^ie secD&d s^lor question cma iean ask: bas to vi^ th^^ ^ 



eff€rtl^eaess of ttie IIW «ercis»^n teaehinf new usmb how to do biilt^^ 

; tbat tbt user has no^prevlQi^ diractf ^peri^be ^^fc^^^^vlnfr 

tt% 1 u^tillzss the capacities of IXCA as both ijist^ctor and assis^an^* , \- 



^ ^ -I 



.4 



/ 





•2,1 Initial Fomattve* fi^mlmtliMtt _ » ^ ' . * ' 

i 2 #l,.^i^^ Py3egt ^taff *_^^^cer one ta3/ti#©> teC 'Mte - r:::! 

a^irationalf ^ffld on^«m ^ongoing ^Basls ey^r since the , parlous ttefelses were 



ertei tp rertew sodificatwii the projert staff, AlJ, ae^^rs of: 
^ tii^-ataf f have'^ent time intereaetlng^th^IIDs^ as uaers iji^-orier ts look 
for SlaT^*©r wayi tO' i^ED^a &e ©peration of the eysti^* 



- . 'jlll*2 Cpiroutar scl^ce mapors , Mfii^ the first week af Har©h 1979^ 

. three undetgra^ate students vhq^^a^tf^jptoe^ bf the Hai^^ati^ and: GosputlBr ^ ^ 

.... * .y. r ■ . ^ . ^ 
^Gieace Cli^ at ^^al tested 




for eancep^ial and progrjtoting 



flaws* tfha^phasis in tois testing.^^s iqpon «ereise^ two.^ None of toe 
three Vttdents h^^^ with ^--line aearrfiing, but^ai^ of 



V 



thwi had had extensive ^ ^^erienoa" working with QO^utarT.angukges and systems* 



1^ 



They wera given a half-hour ^©f Instruction in the basios of learohing and 

........ . _ ...^ .... ^ ... ...... 

^lof-^he^^ctoA -^stCT" a^-ltwas^^^^i^ 



This^ ins true t ion Vas givan infomally by ©ne ©f the p^jeet^'s r^earch / % 
assistants who was afl^^perxeneed searehar. Next, tiia ^tvt&^t.ras l^d^i 



oh to IIDX (in a3^^roisa tw©) ^a^^ehallenged to try t© find, the rlawsgt^r 



try to "bEeakT^yie systemj. 
c A£l threeiof toe stu4emts were able to search sucKsefQUy using IXD&# 
M Ve 



an4- ^» irtto- 



few problw^*, "All three f^iftd a-prograroto 



was subse^enmysgorEerted.^TSl^e consisted of negleqting to reset m 
array m one of toe dii^ostic programs bacH to Ato at toe appropriate 



tima, 'Thm mizm ©f the array eventually overran toe available- s^©raf a space 



durijig-the oourse of toe searcji^ and toe progrKn than bmi; 



from the systain prtoaturaly. 




ERIC 




= %ren# for the oDp^ part^ favorable* Ml thrM vsi^ iapresssa vito-^eir « 

OB^ltoe saardiing* also s^mmd plused wiUi llDA's reaptoMfeM ta = * , 

: saarAer input* r One sear^er, %ms ^rtieularlii^ i^rassad «.th - * 



^ assisferoce in -gatting hrm started. After- toe inittol ffiCm cemand {%^ieh 
IIDA p»dmpts toe aser to mktmw) f the uste' did not few what to di^ iro :its 



7 



* *f^r an ^li^t^ of m»^tatola tw^mds^ ent« «yftelp* * ■ ^fi^/ rtsdmt ^ 
did so # and received ^ea^ugh asustanm to parfanj a satisraeto^* search. 

• V •* ' ■ .... 



^ ^ totf was that tte er^r massage stating^ toat the t^er was not responding 
, quickly enough {€^e## that toere was an "^^ssi^e" delay t&e^l^toean 
eomiandsy mm vBj^ annoyyigi th^ thrf^^^d toI^ for the ^^^^ Tf^-f*- h^g * 



sinoe be^' eKtendad, seoond «iticim^Mde bygone of the stuiSratS/ 




that IlDA t^ded to be overly verbose* The student suggested in * 



fe^rtieular that ttie «ssage ^^^liig the user ttat he will be e^eeted 



to 



rate t he r elevan ce of aU rego rds viewfed afte r a TYPE oonmiand "should hm ^ ' 

stated only once ratiier .than aftor eachlfWE ccnoalnd entered^ . ^ : ^ 

^2.1.3 x^Raal* searghes .^ K Aumber seazdies done by real searchers 

' . . \ s ' ^ • ■ ■■ ' 

tiiroufh eKtoelse in order f or to look at Uie responses * 

..'^"'"^^ ^■""■"/'"^ -■^-■--■"---^^^^^^^^^ - - ; - ----- V ------ 

of tiie sypt^m to *real" seiches* T^ seven^-tw© sea^hes Involved were 

i^akeA frpffl the set of sear^es .^collected li^, ^ study on saM^^ process 

ass«^nt deseri^^Steiefly ^ ^ an earlier report (1), ani in much greater ^ " ' y 



de^h^~FtatcEirn[?rr 

lasiwlly^ toe seareties iirere done on eight differ^t topics ^y^several 
y jouroeyman searghe^ft who reaeived some oi toe problega by Wilyi dfd the 
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jt|p:.r ; s| S€»ching lAing^&eig oim tegamle^^ and return^ seaEdTtransw^^s / 

--——:7r^^^^^'^^^^^^---^^;-^^^ .. ............ .... ...... 

* ^ all intotoeM^f project staff were ijivolv^ in loeklng tor flara that * 

needad^tQ be aorrarta^. In addition y 411 a£ the staff sie^er^ were jibla 

.1 __.._.*:r..__L_*:.LA ^. ............ ; \ ^ ___ ^ :. .. .. _ ^ . ._ . : 



^ gmin f^Bt^h&iid m^rim^BB wito imv IIM would ha^e tiieated tte i^er 

: \ " '1 . . .. .1* ■ ^ ^ .... 'k. V : 

Had dpne the seat^ * Wii la no loa j or s tan^ive changes we re'^^ Mmed- 



.1 

iately made in tha^ aystam as a result of this zeroise # it was pf partieiilaF" 
benefit in insuring thmt the entire staff'^ad^a eoherent and eombtate, feel 



fop^ the pwjeet ^^a whole rather -ttan for just ti^at s^efipon^^^ipon whi-ch ^ 



^ 4m ^helping to provide a basis ^for iatar decisions vario^ guestioris , 

• >^|j.oh arose about* ^fteh* issjies as ^ how and ^at to topleni'mt in the way of " s 

— masaagafr- f rom *IIO^#p#ta. Fi^ 

" ' ■ , - 1 Y _ / / 

^ Important badkgroimd^onteKt agatott which to weigh Var^oi^s user epnmants "j 7; 

. ^. ,„ ^ ^_ - *s _ _ ..^ ..^^^C^ 

^collarted at ^e tiae of use or during ppstVuse Interviews and "di^soussions. , 

' 2*1*4 Library and Information Scienfae familty and (graduate etudente * 



™ ' ^ , — ^ — ^ ^ — ^ 

Three graduate students md two faoulty menders of :the- Drexel UAiversity 

Sehool of *Libra^ and Information Science also went ttaough the entiore set 

a# CIHA exarcisw (axeroises one through three) before ^testing w#s begun 

: _ with noyice! .user^ .^.^ ^jbhese tests were 'Up^ful in that _they^/^^^^^^.\ 

imcQvered s few prograniming errors whifch/had not yet been diecovered and 

corActed. ,^11 of these usars ^were .oxpariancad with on-line bibliographic^ 

" \ - ' ''' i f'^. \ ' \ - " , ■ ^ ' 

searching^ generally l^fd irie IIDA progrtoi* However, these us#rs were 



. not reaUy' useful in *detedting p^oblenis with the content of the instruct 
-\ tional progran from the novl^ce user's point of view*^^ln later testing, 
* ^undergradt^te erigin'eerihg students' with no library training or bibliographic 



^.--r.--. - - T^-r,^-, r-# - ^ -j, ■ .r- .."Jjl,. .- - ' - ' ! < ~— " .' i - , ^ 



,_Ili^lpok3Jig_at_Ui#iis5ue_^ 



assis^aA Airtzig m^^dht ai^e ^al^Uy ranoazned with umm^m lAo hava % 

^, . . ... ... - , . . . ^. . ^ ^ . - 

already had a raas©nabla*dafraa of saaroh training # and oona^uently are to _^ 
hm mxpommd to IIM- only through toe assiatMoe moda. In an aarliar report 



(1) # several ^sslble studies w^ra dasoribad in order of their priority. 
All of toe atudiM var^ in pa^t shaped ^ o^ilderations of toa aymil^la 
reeo^ces ior conducting posstole studiaa^ For ^ampla# givra that uaars 



could only accasa the ayat^ one at a timer any uaer group had to be avail* 
abla for sore than^^a- session # tee much -available ^^up-%ma aompos^ of - 

- - - - ' - ^.^^^ - - - , S -_ - - 1.|-t_ ^ ^ s - ^ ^ - ■ - - , ^ 

-new gra^uata studanta in tha Drescal Schsol of tibra^ and Information' , : 

* : . » - 

Soience. . * 



2.2^1 FuridamCTtals_Qf Librae .md lnfejmation_Science uaer group . 

. "... ^ . \ ' . - ......... / ... " ~~ '.^\ . . 

Each yil^ ^tfi^ School of Library and Infomation Scianca at prexel Univeii^ 
ateits a number of epw ^aduate atudenti^s. Whanjtheaa atudants ara aAnittedf 



thay are requirad to r^iatar for a coi^ae antitled^ "Fmdaaentals of 

Iiibr«y> and Infomation Scianca,^ One of ttia cOTpohents of this course ia 
J* . * 

la block of instruction in computer-based bibliographic aaarching, Thi# 
block Q l_JLnateuction^_Jfco MIing^r^ghl^ tfairtean hougM^^J LndLud€j_bo,th_ 



claa^oom lecti^es and hands-on laboratory axparience in searching. TiCm 



'^idammtala^ atudante were racOTaended as a group for study not, only A. 



- ...v. ........... v:. = . . vg. 



at a novimm Iw^/, fairly teri&gMiTOus In teatain^ and «perleieer aiid 

%roiild trigger the dlagnoetiai^ ^ ^ . . ^ ^ * 

vplOTteera from the mmmmv tem 1979 "Pimdmental^" elass« The projeet 

^ . - . . , ■ .. . -. . . 

iggaber^ who reCT^ittog^trea^ 

gai^ additional ^perienee in searching* Tmm a alass of twenty- six 
students * there were twan^-two TOlunteers. Of these twenQ'-two, sevantaen 
actaally begM tha study, , All df thosa students whp fiegan completed^ Eagh 



stud^t was asked t© asnduot two aeardies- The* same to?© seareh requasts 
■ ^ ^ were given^ to all off tha students in the aarass order r^Sinoe 'soma of thaaa^ 
\ stndenta may mlttoataly go on to beooae intersadiaries^ the statement of 
the problem waa muGh like those that they oould e^ect to reeeive ^oept 

- - - . tha t-.in-t thls -gMe >^ey able ~ to ' interaat ^with the per^n stjgmit - 

request- The requests iiaed appear in Appendix A. ~ ^ 

Randondy/ , one-half of the usera were assigned to eonduot tha two 



searohaa with liDA^assistanoe^ whila toe ©tl^r half were assi^ed to qonduct 
the sea^ohea without IIDA assiatfanoe'*^ In Itoth^ases/ thm searching waa done 
through the 'UDA syatem sa that search records ware kept, automatically, but 

. with the n gwup^ the^ interactiva„ inessagem..wera- a^pr assed 

so that the users ware not given tha IIDA feedback. None of tiia users in 
this group were exposed to tha instructional programs. The randomisation 

.was accoroglished by making up# in advance /^the randomization schedule^, and 



, then assign ^Br each atudent to a group according to the schedule as that 
student" appeared at the laboratory for his or her first ippoinjOTent. 

r - t ......... -. ... .- .... . ........ ....... ...... ^ 



; / . = n to ymgn^r OTBDands md siaterl^f i^dh they- had' learned in eXaMs- 

f 1 lOOT iMmimmtipn* -Shmy ware alsa infamed tiiat they ^uld >ase anytMng 



lii^ hkd learned in t^ir class ^ bitt tiiat^fiie kmrnimtAnts were npt ^le to 
ac^ise tl«Ei ahQUt what would wofk ©r how to^ ^r^lment partietilar TOmiands* 



TOyose users who searched with IttA^^istanoe were si^ly a^isad to 'do so 
rather* than to^sk questions of toe aiiii^mts. In order to t^ to reduce 
-the^pogBibi^ity-of - a^inee preparation on- t^e part- o^-ti^ 



eaA of the jasers was^asked not to disOTs^ th^ssar^^ topics or their, 
searohes witii their olassiaates imtil tht entire stu^= had been epnplet^ by 
^ all of ths participanta , Most of Uie^^^^^ers'' did their s^rehing in two 



half -homr sea eio ns on fe#o differ^t days. ^ * 

(2) Resttlts . ^ Although seventeen students took part in this 
* ' ^ Btu^l toie of UiOT Will be rfeported* Q^^-^g — - - 

systeta ^di^f ioulties enMtmtered dwing toe simtter pilot testing of IIDA 

involVi^f^ M ^^ilnyo ¥f\ j-y-^nrnWrnTr .g/^fna fif ^hm mgghjji^ y^/^gj^A ^tlj^^n^ T^^f^g^ 

from tOTporary memory storage, Thus^ .toese reoordi were witten over by 
naw'« wer s records ^fae fore ^h^^faad been ^ o utput TJt^ hard wup ^ orro™^fiito"loTO™™~ 
of records ^d not a^ear to be statistically^ related to the individual 
user's group mefflbershlp, ^ ^ ^ 



r 



The-ieleven user^ discussed here are those for whom the oadiine resorded 
^agnostio data were available for ^toe*seaeaid pf the ^bfb searches* Basically, 

» ' i ' 

fomr kinda of data will ba diacuased below. These ares (a) dMographic 
information colleclbed on toe pre-taat quastionnairasi (b) the frtquencies 
^ith-iAioh^^e-various=oat«goriee=of--«M 



(e) various attituda^meaeures OTllegted on toe post-- test questionnaires f ^ 



and (d) dasctiption of the remarks madp by the IIDA users on the post- 
.test questiomiaires . The questionnaires used for this study appear in 



AppendiK B, • 

■ ^ : (a) Demographics. In terns of their d^ographic character- 

istics^ the eleven users all had a bachelor's ^pgreei and none had m\y 
advanced degrees.^ fflnong toe sIk females and one male in the group which 
- used IID^ assistances there were two students who indicated having A degree 
In Library Scl^ces two who indicated a degree in the^ Liberal Arts# two 
the Social md Behavforal Sciences, and one. in toe Biological Sciences* 
Amonf the' five v^men in the group who searched. without IIDA assiftance.# 
thete Was one with a dpgree in toe pKysical Sciences, and the raat hai 

^. n ^ : V 

Liberal" JUrts degrees. Looking at undergraduate course worlg in terms of ti 



in 



two clusters* of courses^^one cluster of courses being those in Mathematics, 
Computer Science, Physical Science and Engineering, with' the other cluster 
* , of GOUTS es being- those in toe biological ? Social, and Behavioral Sciences^^ 
thfera wfre no significant differences between the two groups (Mann-Whitney 



<9 



U ^est). 

In terms of their exposure to computers and bibliographic searching, 

_ I 

the two groups seemed to be quite ""similar in that while tV?o of the stude: 



nts^ 



in toe no^IIDA group were the only- ones to report knowing a computer language, 
the two groups did not differ in describing their computer skills as vir- 
tually non--existent (Mann-V^itney U Test) ,^and none of the users reported 
having my experience with computer packages such a/^SPSS or SAS, All of 
theL^ajrticipants except one in toe IIDA group reported having done some on^ 
line searching before begirming the study, Whi^ t\m of the five users in 
the no-*lIDA assistance group had had someone do on-line searching for them. 
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three af the seven in the IIDA assistance group had had someone do * 
learahing for them^'none of the ulara indicated having it done more than » 



twlGe- . i ^ 

a 

(b) Diagnostics, For those usets v^ere the machine records 
of the nus^er of times various diagnostics were triggered were available # 

data for the "two groups are rejpor ted in Tables 1 and 2* These diagnostic 
prooedurSfefall Jji^ three broad claaMfeai a^) Syntactic* *b) Local Context^ , 
and £) Global Cont^t,* For tile first category, the concern is with the ; 
validity of ^ coEmtod^ amd this CQnaideration is .entirely_ cont^t-free* / 
The guiding operational principle is Jdiat a conmand is valid if and only 

if it would be aceepted by DJMiOG. - ^ 

* , f 

^ Given that a commjtfid satisfies the context^ free structural ru^esy 

• ■ ~ * *- 

there ^may yet be problems that r^ge from ^atd. errors (such as the' use of 
an undefined^set number in a COMBINE conmMd) , to mere inelegiuice of usage 
(such as repeating a previous comnand) • ^e contesct of the analysis is 
^ li^ited^ and the commands are examined in the context of the accmnulated 
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history of previous conmiands. For the global context diagnostics^ the 

concern is with a set of commands as an entity rather thm with individual 

coamiands* The non^syntactic diagnostic procedures = are actually implemented 

as a series of rules ^ not unlike rules in a formal decision t^le. These 

various categories of rules are described completely in a previous 
■ - 

report (3).^ . ^ ^ 

Comparisons of th& two groups conducted using the Hann^.Whitney U Test 
revealed that toey did not differ statistically from^Sch other in terms of 
the frequencies with which the various categories of diagnostics were invoked 
during thm process of searching. There is, however, on. toe total number of 
"errors" a difference which is of borderline signific»ce (U - 8, p - ,074, 



Diagnostic 
Category 



1. 
2. 



SyirttaK 

Conroand 
Repptition 



3* Uninfonnativa 
Fonnats 

4* Null Sots 
Craated 

^5* Unused 
Sets 

-ft^r-MTim : ^ 



7* viewing Requ^^ts 
jExcassive 

8, String/C^cle 



9. Thrashing 



10^, Dwelling 

11, Local -(2 + 3 + 
4 + 5 + 6 + 7) 

12. Global (8+9 
+ 10) 



-0 ^-0- 



1 

0 



1 

l" 



0 g 



0^ 



0 0 

^^^^ 



1 

0 
0 



14 



Total (1 + 11 
+ 12) 



24 



Table 1. Frequency oC diagnostics fo Np-IIDA Group, 



/ / 




Diagnostia 
Gatagory 



fi- 



5« ^uaed 

.Seta 




Syntax 

Gommand 
Repetition 



t^informativ© 
Fomats 

Null Sets 
Created 



7. Viewing Requeets 
&ccessive * 



a* Strihg 
" Length 



10* ^ Dwelling 

11* LQcai (2 + 3 + 4 
+ 1+6 + 7) 

12, Gldbal (8 + 9 
+ 10) 



T^tal (1 + U 
+n.2) 



Table 2, Frequency of , Dia^ostics foe IIDA Assistanga Group, 



one-tailed) , and which suggests the possibility of IIDA "assisted users 

making fewer overall "errors." While, ijt is not reasonable to tryitb maka 

- ^- --^ ..l-.^,,^^ . . . . -.—i .:.- .-.-:.. 

much out of this finding; it'*is ^t least reassuring '^to find that ^the IIDA 

assistance does not appear to make seajfching mote- difficult or morf 

complicated for the .user. - * 

(c) Attitudes* - The only differences beta^een the two groups 
found in e^^ination of the attitude measures on^the post-test questionnaires 
showed up 'on two questions', ^mile the IIDA assisted group showed a higher ■ 
level of agreement with the statement that the assistance mode of IIDA' had 
helped in searching more effectively (Mann-Whitney =^ 4,5^'p < .05), it 

was the no-XID^ group vrtiich indicated the highest ^degree of agreement with 

i -- 4 

i 

the statement that the IIDA searching had been enjoyable (Mann=Whitney 

U ^ p ^< .05) . ^ , * ^ 

• ' ^ ■ / 

(d) Post-search corranents, Wien asked to give thei|f ^sitive 



^ impressions of IIDA, the IIDA assisted users mentioned a wide variety ^^f 
^^things-^th#r than^ all^^greein^ points, -'For example ^-one- 

user appreciated the system automatically calling TY^WET to4each DIAJiOG, 
. Another found the syntax error criticiras and comients on seatch' techniques 
^ployed (e.g., thrashing) to be helpful. In addition, participants 
reported liking i "the well-explaihed introduction and conmands," "the 
clear and natural language," "the usefulness of the help mode as well as 
the ability to evaluate citations," and, "the indication of errors and the 
need for assistance*" 

, l^en asHed to give toe negative impressions of IIDA, a few of the 
participants reported finding tiie instructions unclear and complicated. One 
user felt that tiie vocabulary ras too technical and overly wordy* Another 

17 ' 
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disliked' I^DA's limiting size of the^et of ci^ationi thafc Gould be . 
vie^Ji^^ftth a iingle TYPE =«imnand, A few of ^hosa users , who did not searah 



witri IIDA assistance TOmplained of the aack of help and eKprassed feeling - 
frustrated and unprepafrad, v ' ^ 

'.Ml Nona ef the participants reported anything tfiat ^hey though* should 

^ ' ^ r ' , ' - . . : 

have»beam working in IIDA'vrtiich was not* . Whan asked to describe situations 

' - . " ^ • ■ ^ ' ■] . - 

' — where IIM did the wrong thingi a few of the partiofpitfita reported instmeas, 
, ^ of minor ^stsn , flaws ./ tos^t of theSe turned, out to have been ^alatad to 

phone jjffiim action pr^Weitts. ^ ^ ; 

Uppn being asked when they felt they kne^ enough to search without J 
IIOA'a aesistano4# the users gave replies ranging from^ "before beginnir^^" 

tOr "^tS^ toe second search," One usfer reported, "when my results came \ 

- ^ ;V- ^ / , y- — — — — ^ — — ^ -_- ~ — — — 

out the waj' J ^peated." when asked to ^plain their reasons for feeling «^ 

tiiey oould* or could not seatoh withoutrtlDA assistance,; a few users felt 

fcthat tiidy were net ^iainiliar enough with searching aiid neededvffiore experience, * 

-A few cithers f^It/timt the/.were conftisad hy iiDA's 'Jjistructiona, and did 

not learn anoufh" to search withou^^ 



U** felt she no lo^^er nefeded assistance because the system was^simple to use^ 

and one'^reportad feelii^^^ OT^d search without IIDA becausa of prio^ 
experience* ^ ' 

(3) Discussion* 'The uSers described in this study wtte all 
trained but inexper|.enced searchers* some of whom did searches with IIDA 
> assistance* In general^ the reactions of those^ho did their searches with = 
IIDA wire positive in that toey felt the syst^ had provided them with valu-- 
BhXm px worthwhile assistance* ^^il^e there does not appear to be My ^effect ^ 
of IIDA ^sslstwce Tjpon toe fr^imcy with which the diagnostics are trig^ 
ge^e^ diuring a search* the number of searchers in each frou^ is relatively 



small, and bq thore map^ba effects which have not been detected. It, ^ ' 4 
reassuring^ howav'er^ finE that It apoears as though the direction oi 



f 



? 



differCTces would favor the [liDA grOup.Tf the results were signrrfcant* 



This would seem to make it reasdnable to asswne that even -if IIDA assistance 
does not measurably improve the searche^^^ performance^ it also does not 



seem to impaly it, ./ . ' f^^^; 

2*2,2 Teehni car writing ufeog ^^rQup *' - ^ , , » ' 




Turning tq the issue yot hovf^^ll IIDA i% able .ta porfbrm the service^ 
of .enabling tlqJluq.. use^s .to do a sucebsiful search , the concern' ^s basically 



with a usA' group which has had jio previous experience^ either direct or 
Irts^ruttlonal, with eomputeriied information retrieval. One such available 
gro^ ofiAisers was found among^'Drexel undergracluate^ngfnecring students* 



HDughly four years a^o^ Drexel* University !s Collegir of Engineering insti-- 



tuted a course requirement in technical writing for all ^ -^glneer ing students 

Each term there are several sections of thip course oxfered, * Several of^ 

. * ' ' ' f - V • « _ ^ 

tne faculty involved in teaching this course agreed and felt' quite strongly 



that it would be a relevant educational experience for the' students \tc 
4larn somethirig about bibliograpMc information retrieval. Many Drexel 
eagineering gi;aduates will be empioyed by organisations-^ which utilize the 
services 'Of information retrieval specialists', Pres\ffiiably students who have 
had ^some direct ^^o&ure to ^searching shpuld be better able to wrk with the 

' ^ / \ * 4^ * ^ ' 

people doing tiie searching - ' , . * ^ - ■ 

As this study was origihally planned' (15 , both the pedafogical and ; 

- - r ^ ^ - 

escperiBientai requirements were to have been satis tied by having each student 

. ^ ^ , '' 

perform a sewch on a self-selected tppitt *iti each of , two ways-.^ One method 

would hav^-been to have the search done through leamuig to do ajid actually 

doing the search through IIDA, The second metiibd would have been to have 



the search .dofte tiirough ^the nore conventional pracess working with a 
% . * lArary intermedial vmo would have actually cdnduct^ the search. Each 

- n., 4 - : .. ^.■■,^.■;;^v..■ . . .. 



c ♦Student would haVe bem randdi^^ assigned either to do the search f ifit . § ^ 
w^th itoA, *'or to dd= the search first with the ^intemediary* This would. • ' ^^'^ 
^ have allowed compariyDns between "^the searches ddne by IIDA trained' and * *^ ' 

^ aislited users # and the searches dane by. a eingle previously trained and 
K . experi^ced sgarifher. ^ % ■ - £ 

^ This study design was abandoned, however, aBout a week hpfore " ' i 

begisWiin'g of its pltoned start in the suraner t^pa of^l97#^ Because of a 

- - - 3 r _ ' * * 'I 

«mplicat#d^ mm% of e^ipment and jcheduling prqbl^eins, and the resulting 

shorbage of access time ta^ DIALOG, tit be^aine clear that it was a physicrf^ 

\ ' J \ . \ ' ' ^ \ ^ ' 

, ' iraposs^ility to conduct the study as,jglanned. Howe v^i, con tingenGy pl^s 

— --^ ^ -:- ^ r--^- - - - - - ^ -.^^.^ ____ , ..^^ _ _ - ^ -- * ' ^ ---- 

had been prepared in ad\rance, and afte^ consultation with tha tachnical/ ^ \ ' * 

witlng instrtibtoti f the project staff, and the project evaluation 



i 



consultanj^, ' the decision was made to dimply provide a^nto^er of the stud^ta ' 

with 11^ training and search assistai\ce* Thus , \^e^study r^porte^ below^ r ' ' ^" 

^ * * L , f - . 

- is deserlpt ive ra tfeer ^#ian - ttperimen tal^ Ih 'tia tuf e ^ .3^-..: . . ..^ ..-^^^ 

\ (1) PgQcadure . In all, thirty-five undergraduate student! '"ware ' ' 

J contacted through two sections pi the technical. writing class in toe suimer 
teCTi of 1979, Student participation in ^e IIDA training *ras strongly 
^couragedTby the instniet^rs of the two 'sections involved. Participatiorf ^ 7 

i i ' ^ I , " 

was ifot, however, a class* requirement in tha sense -that a sttadant wo^d be 1^ 
.^aded negatively far not participating. The IipA training was' ifttegrated 
W3.th the technical writingf course work in that tha rasults of the s'earching ^ j 

were potentially useable for the^^udents in Drep^ation of required assl^- ^ 

/ / . . ' ■ 

iifnta^for the course. A syH^ui for tie coittse appears in Appendix 



- '.^ -l -- 

/ -"IB .If 

nally intended that thp students wuld complete the IIDA 
'training, and/the\ assisted Is e^ch of their choace, in two two-hou^ seasons , 
"^do^r^^^^S'^fTtK^ 

^tebefltled by mutually a^^^^ ^pop appdintments during the avail^^lG" periods 

■ * * • ^ ^'^'^ \ ' . ^ ' . . - * ' - 

of access to tih^ equipment And the '^stom, with th^ scheduling beings kept / 

as flexible 'as possible to' ^combdati thf* gro^l&is ofi Efud^ts with jobs, 

/ ,^etc. J'or*' exajnple, ijany sjNsiona ^wore scheduled ii\ the 'late afternoon, 

early, evening, and^on Saturday^ morning* ^ . ' 

When th^ students reported to the IIDA lab for thMr first appointinent, 

^ \ ; ■- ----^ = - ■ - . - — 

one the thtee IIDA assistants gfictfted them and as^ed each of thept to f ill^ 

' . - ' % \ * ^ ' 

* ■ ^oufe a pra-training questionRaira which was used .to^ collect basic data^ about 

^ . . • - ^ ■ , ^ ' • , . ' , , 

* ^ tha user ^oup. All of the questitffrmaires used in this study app^r in 
^ ^ Appendix D. N^t, the assistant /logged the atutient onto the IIDA training 
^ ■ (jpjtogramT explaining ^hat the first Zeroise |t\qjluded an introduction' to 4 . % 

arfd basic instruction seirchin/. The student (was also tol*'«Tat he or ^ 
she iho^ld cpfiitinue on after Completing exardise one, and do exercise two,' 

^ \ J ' ^ ^ ^ S . 

was to do" aitfieir a^aarch of his or her own choice within the constraints . : 

" of exarcise two, or to do a seatch on one of a set of two topics which were 

I Y provided byj the IIDA staffj^ These tw6 searches are A.cont«inad in Appendix 

When the second exercise had baan completed, the student was asked /to fill 

^ , out the intermediate questionnaira* ^ . \ * 

^ * The second appointanent" followed a similar pattam. €%ercise three was 

introduced as a contintiatlon of e^cercis'e ^e arid of more detailed instruct- 

tion* Bcercise three was followed by "'zeroise four" whirls tha assistance 

i^de of IlDA^y and which gives the user the opportimity to do an unconstraijied 

search on his or her own topic* For thisa students who did not have a topic 

"""" ^ - - ^ V 



M _^ that- they partieularly w^ttd to search, a list of "graduatiori ^ercise" 

. .-M ' * ^ s©8L^^hgs was provide^. These saarhhefe are fisted in*Appendix F^y ^Upon 



\ eompletTgn*of exeraise fouTr the user was asked ^to fill out the post- 

' /_ { • = 

training questiomaire. 

• . . ■ ^ . ■ • *i . 

Ditt-ing 411 phases bt the searching exercisas, the noA assistants 



• iirged th^ partieipants to utilise the Help faeili^ies of 11^ rather thaji ^ 
to try to rely the aesistants to mswer questions* SOTie ^partloipants «' 
found Kibliog^aphio on-lihe searching so foreign to their experience^ or ]j v 
wsM' so fcesitant about working wit^^ ^ computer, that they needed help wi,^ • / 

I the BEGIN eommand*^ In thisNcase, the IIDA assistant w©ul4 guide* the user. 
Further / somi .p»ticiptnts did not remember all of the commands* *When a ' 
user would ask how> for sample, one cottld'view some ^th4 sets created, ' ^ * 

* > * ^ ^ * . ' ■ , t * j ■ 

^ A V_ - --- - . . - - - . . X - -- - - - . . . ^- i - - - 

^ '^^^ tlve JIDA assistant would, remind thp us er ' of the TYPE command- However , if * 

^ the User did not* know how to use that command, 'the assist^yit would advise 

the user to "seek help. Generally, the assistmts would ^luriteer infor-* 

mation* on such topics aai how-to* correct a character -or. cancel a line of* ^ \ 
.i ^ , ^ • - ^ ■ ■ 

^ ^^"^ " "^"#rtnt7^ how^ti r 

tor OR while ' combining set's , ' 

' • i ' * • ^ . . ^ ' ' *^ " ' 

f% Occasionally, a participant was topped £rma the system as a result , of 

. " a failure either by the DIALOG con^uter, 'the MuLtics computer, or the network, 

SQim this occurred toward the end of exercise two or four, if the user felt 
r exercise was completed to his or her satisfaction, and if the I IDA 

* assistant agre^, .the^ that ^ercise was said to be completed* Generaiayi 
however, iji complete exercises were re-stMted, No.studmt was forced or 
_ coerced into toing any of the exercises. For eXOT^le, one participant chose ' ' 

not to do exercise three af€er hearing classroom nmors that this ^ercise 
was boring* ^ He felt that ha had seen enough through the use of help in , 
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exercise tvo, Aijothcr user quickly fibred of exercise three, after com- 
pleting only two p£ the eight frames, and want on inftbad to begin hii 



search in exercis 
asked many questl 



e four. ' It i^s interestingr to noteV hoyever, that he 

i ♦ 

Tns which wo^ld have been answered in fexerci&e three. 



Also, he, along with a number 'of other participants,, setmad reluctan^ to 



-use the helppfaci^ 



ity. 



(2) Re 



ul%s . Although inijkially thirty-five students were 



all four of the ex 

4 

and TO dropped par 
there was simply " 
"F^Rk dawn- <rQn ^ Bl , 



originally recruited for this study, "thirteen dropped' out or Sid not finish 



grcises for one reason or aftDther,, iVo dropped the cotose 
:icipation^in' the study. Another two indicated^ that 



oo much hassle" in getting' on the system and having it 
i d th at he. kad^enough I nfomation from the second exer- 



oise for puiyoses of tte course, a^d another claimed to have no topic she 
wanted to search- ■ For the remainder of the dropouts, there was no reason 

C ■ • I ■ ■ . ■ ^- 

given* Usually the^' would just not show up for their appointments or 
would call in with some reason for 'being unable to keep an appointment* 

Based u^n user comments, instructor feedback, and a post-training « 
debriefing conducted with one of the technical writing classes as a group, - 
the high ^dropout rate is attributable to two major factors, the first of 
these was the length and amount of material covered in exercise three as 
it was structured for these u&ers* This experience led to i^tnajor restruc-- 
tiiring' .ihe exercise* The second factor contributing to the high dropout 
rate was a relatively high frequency ^of system difficulties external to the 
HDA prograffii, Wiile occasional probl^s wete encountered with the auto- . 
call device, ^hich placed the TYMNET phone call to the DIAIiOG^ comF&ter , more 
often there were probing with DIALOG going down* However, the single most 
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'%recpiaiit set of profaleins arose from network difficulties which resulted 
= 'in several participMts being required to 'schedul^i^re than Wo sepsions 
fh oraiFIB complete the e^cercises* Some 'ufe^s simply refused, others 
diiappeared* 

The eighteen^ uaers discussed here are those who completed all four 
eKercises, for whom- the machine recotded diagnostic data were avail- 

abl^ for the exercise four, 6r assistance siQde, search* llie system 
difficulty v^ioh resulted ift. tiie loss of some student records oacuAed 
' * during a period of time whan/both users, from this grot;^ and from the 
Library and Infonnation Science graduate students user group were being 
soheduled for sessions wi^h IIDA.. As with the Library and" Inf ©nnation 
Science user group, basically four kinds of data will be discussed ^low.' 
These are I (a) d^ographio information collected on the pre- training ^ 
questionnaires; (b)^ the frequencies with which the various categories of ' 
'machine diagnostics were activated; (c) various attitude measures collected 
on the post-teBt questionnaires; and, (d) a description of the Remarks made 
. . _ fez XIOA . users . on ^the .post^t est gqueationna ires * — ^ - ^ - 

"v' ' M Demographics. The eightaenV users in this group were 

all advanced undergraduates with no sophomores and only two seniors in the 
group. Five of thm group were female, 4nd none of the group had any 
adv^iced degrees* My mjor, there, were fifteen Engineertog ^students of 
various types," vdth Slectrical aigineering being the major most frequently 
nMied. Among the three non-engineers, one was a Business major, and the 
other two were majors In Clinical Dietetics. .All of the user group reported 
a pattern of course work consistent^ with the reported majors. While there . 
was -some individual variation, the users reported^^ heavy enphasis upon 
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l^themtics, Spiencep and .Bigineering, =with- most :haVing only introductory 
■ eouries in the Biological^ Social, and Behavioral Sciences. 4 * , 



Wiile^ a lew of the vsc^s reported having no c^puter skiais, the 

av^aga seLf-rating of compiiter skills fej.1 into* the ^category^ j^good 

enough to get by^" Only 'four of the eighteen reported knowing no computer' 

languages. Three of these four were Engineering majors, and one wa$ a 

Clinical Dietetics major. For tfibsa users who reported knowing a computer 

iMguage, the aver-aga number of languages ^known wa's 1,9/ with a range of 
« 

one to five languages* The modal number was one, ^None of the users reported 
^ _ _ * ' ...... _ .. _ . 

havihg any experience with pre-prograiraned library packages such as SPSS, 

With respect, tp bibliographic on-line searching , none of the users had ever 

done bibliographic learching before, and only two reported having had It 

done for thOT* One of the had had it done twice; the other, four times, 

(b) Diagnostics. Table 3 reports the frequencies with which 
' the individual users searching behavior triggered the IIDA diagnostics. 
,The totals fctf each category are presented in Table 4, Because of the 



considerable on-^line interference encountered during much of the sunmer, 
"errors" reported by the computer were not caused by the searchers. 



The search transcripts .were examined in detail by the IIDA staff, and the 



irpchine reported frequency for the various categories was adjusted to rMOVe 
errors which were in fact a result of a technical problem with the system. 
TMoae technical faults or errors which resulted from a malfunctioning of 
the IIDA programs were also brought to the attention of the system progr™mei 

i 

SO that future repetitions could be minimiked or entirely avoided. A dis- 
cussion, of tha re^icoring procedures appears in Appehdibc G, 

(c) Post-training Tneasurfes. Table 5 reports the in,ter- 



correlations among several variables from the post-training questioApAre 
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total 



Average 



- - :/ y . - 

y Rep^titien ^ 


.82 4.56 *„ 

' . M i. ^ ^ ■ ' ■ ' . ' 
8 0.44 




f Formats 

1 4 Null^-Sett--— 
*v Created 

Vmim ed^..-^ — — ^ 


0 0.00. • 




24 . 1.33 ' ^ 




i Sets 

v- ^ - - 

\6. Time 


4 ■ ' 0.22 
10 0,56 






7* Viewing Requests 
'8, String/CyQle 


19 1.06 


Lengtii 
9, Thrashing , 


21 1.17 
0 ■ 0.00 


^ ^ *V 10- Dwelling 
11, Relevance 


0 0-00 

61 3,39 ' ^ 


12. Legal (2+3+4 
+ S + 6 + 7) 

13^* dlobal <8 + 9 
. +10 +11} 

. . .... ... J ... , _ ^- . . - 


Is 3.61 

79^ 4,39 ^ 



TOtal <1 + 12 
+ 13) 



229 



12.72 



TirtJle .4 • _ Totals and Averages . 
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and &dm the nianhin^ r#cofds« One ©f tte ^Mtlom ©a tiie post^-trainlflf 
asted^^a^todaa*^ to report th^ 



tT: " ratoi ayi^ in-^fe getteh^timt were *^ery us#ful,* *Usa£ulv* and 'Wselsss.^ 
» - Sjace tha ultJmte CTltariQn for the ^lue of tte Infoa^t iQn retridved, 

- li#a in t^a judp^nt of tte ^er|. tiiis %ma fait to ba m partiQUlttly 
- ^is^rtai^ index ^to the muacmms ot tha systCT. The average parcantage ; 

repor£ad for the wafnl w^^ry TO mth a wing e fMaa 0*0% 



80^0%* The mseiul oatagoj^ raaged^^^i^ to IOb#0%# %d.th an average / 

of" 34 . t % The -UBalea a ^-oat^ory- ji^f ad - f mm O ^ 0% to-^100 * 0% |-btit-the— = — 



average was 39,6%, Tta^V ttia average persentaga of usafulJjtCT^ was just 



over 60.0%# irtth a range from 0*0% %a 100*0%, For purposes of si^plifioa'^ 
tl onj t he variableVrepprted in Table S is the total per cantage uBaful^ 



Also repoaftad in Table b are tha interTOrraiations o£ tna various vmriaislS^ 
with the thrae sajor oatagorias of dlagndstio faults or errors. 

The raiaaining 'rariablas in Ti^le 5 al^ oose froia tiie pist^training 

*■ 

questipnnalre. On the tiiird page of this questionnaire/ the students 



ware asked to rata the dagrea to whieh they agreed or disagreed with the 
idea that the use of 11^ ha d Jbem eaalf r t han e^^eetad. This Itam was 



f . soored with one being "strongly disagree" and five baing "strongly agree. 
Thm average was 3,00/ witfr a standard deviation 0*14, Another variabla 
ftos tills saaa page asked tha students to rata their degree of agreement 



with the statOTTOt ttiat tiiey would reeOTmend that others ^Ao want to do 
blbliographle seardiing laam through IIDA* Again # tiiis it scored 
with one being "strongly disagree" and five bejjig "strongly agree*" ^e 
^T^K^f ^ 2 ,89^ with a standard deviation of 0.96. The , third sia jor vari- 
able from this page of the qugstiohnalre asked the students to rate their 
degree of car taijity% that would refuse the system if they had aoGess , 
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Satisfaction^ 



Eaiitf 



51ob^ Faylts 



a?" 



.68* .60* .53* 1,00 

,06 *00 -,16 ^ ,14 '^\12 



1.00 



Ki 
3* 



^ ?7 



.61* '-,SQ* -.45* -,«* «,5S* -*13 
a§* , 5^ * , 5Q* ,2r 



, 07 * 1 , 00 
.04 .13 



1,00 



^*35 -.22 1^00" 



.65 



5 Corrtlationi Betwaan Pest-ttaining Mi^iffes. 



.32 
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to it. Si£s.s^U^e'ms s^red frcaa oa« to five with fiv@ indica^ii^ 
^wapli^ G^ ta i n ^ averagV jati og here ^mm 3>llf wit^ a. 

eeatar of "^thi gale, - - 



three ^^lalniiig vuiablea in TB^lm 5 each ^tpressnt the mim of 



^^^e« i^f f ermt sets of ^ee^tloas from the port*t^fdning quest ionpilre • ; 
ate a^st fix qa^tiona^ otf tJie j^Mtibjanaire ^e atudanti to rate 



^^efa^^'^ gree of saSE ^ otion -^tit h va gionis^^ pe^s ^^^ 

jiiait _cOT^etedt^„_ TO leogpd fro^ one to fQinry %Ath four feeing 



the oost pi^itlve value « siaie^ of fhmm bIm ratli^f \mi computed and 



as a TCasura of the in4Lid^dtaai*s satisfadtiOT with the eaardi 
mpmtxmQm^mm a whole,*:, Bie average total was 16,llj with a stimdard devi^ 



satisfaotion with the learoh process* ^ ^ 



ieries of questions' assaaslng tiie Individual's 'agreMent with positive 



statisents ^out the I JDA search "s^t^* EaA of th^S it ma was scored ^ 
on a scale tram one to five* with five being the most positive or favorablfe 



respdnse about tiie IIOJl semreh aystem, ^e sw of the^e eight scales wmm 
oomputedf and takm as a meMure of the individual's liking for tiie lldk 
search systto* The average total was 24,00, witt a standard deviation of 
-frS O * This g verageH^ wi^aKy-fafttcateg-nei^er^ 



, for# the IIE^ search system* i . 

Pimitlyj the q^stiarmaire al^ cantained a set &f mix bi-polar adjac^ 
tive sales assessing die stud^t's attitude toward using thm catnputer for 
a classroom assignment. The scales were scored from one to seven i with ^ 
seven being assigned to 'tM Ppr each studentr 



tSm iidividt^ s^le ratings -were totaled ^^©ss tte e« salas. The 

/ — -— -. ' — ' ■ -j^'"' -— -■- — -: 7-'..- — . , - ~' — ; — — Jl-i^: .-_"...J- .. ,— 

Prm the d^welations In ^Lble S> it be toat toe three ; ^ 

/ attltuda meaBuras"satiBfa€tion^ liking, attitude—are. highly correlated. 

with each otter.' ^Censi d^rea o£^i^int^«rrelatiQn# and the looa- 1^ 



^ti^^J^eaar^f^Hiiie^meMi^^ 

claases did not# on. the average, particularly like ©r dislike the searching 



and search tracing* _ 

TOere ia soma indication ttat the students were less positive tiian 



"th#y li^ght tiava'^beeh^liy Uie saareh trainiaf and aeare ^ifi g - - 

turaad ou^ t© InvQlve sere work torn ttay had orig^^lly expected* Ihis is 
supported by three bits , of evidmge, F^?^^OTl^r^OTi^^^Wr^weny 
positive correlation between tl^ studOTts' responses the statamen? that 



the use M IIDA was easier than es^eeted* ^ose users \rfio were siore nega- 
tive about the sear Aing and saarch ^parience were also more likely to 
disagree wi^ the idea that searching was easier than expected, Also, one 
set of attitude ijuestlons, the bi-polar scales, was aswng the questions 
%rtiich appeared on both the pre- and post^tralnisg^'guest^oM these 
scalas were'*^^^ ttos tiie pre-* to 

toe post- training ^astlonnaire. On the pre-trateisg administration of the 
sealesV the average 26*9, with a standard deviation of 6.24. 

The difference between the pre^ and post-training scores represents a sta- 



.tistically slgnificmt decline It ^ 2.62, df ^ 17, p < ,05, two^tailed) , 
ll^^SlIy7^^ie^eSiiintsT^e^ 



la the open-ended questions on ttie ^st-tr^jkng qyes^oima indicated a 

'= ' * . 

fair ^^mt ©f disiitisf action irith the momt of 'work re^nped to produee * 
iiiaC*tiS^5^^eHta "^mr&i iM^any Kind off rea^naBle ^payoff* Xn partl^lari 
the n^gatiTC OOTsents focused upon the lengtt tod mount of ^taH^^regu^sd 
ia-^eroisi^-thrae^ — - — — — — - — ^^^..^-^ ^ ^ ^ — ^ 



One _^te«rtinf idea -Aat tserg ea from con0lderatio|k of the pattara 
of escalations in T^lm 5 can be seen by foouaing'^n tti«e sets of 
porrelatlona'^ First# note the soderate positive corralatione bewem the 



thjbe attitude ^asures and the percentage of useful ^i tama retrieTOd. 
N«tf note ttS 

errors the user made and the percentage of useful it ^e trie fin a iUly 



note the asodermte negative TOrrelations between the n^sdser of syntax errors 



h~~,-^-~w- — ana-tht^V3^om-^ttltude-~TOasur#s^^ This pattern suggests- the-- p^eibi 

that a strong ccmponent of the students' evaluattcn of vairiou% aspects of 
the systOT was determined by tiie perceived utility, of the information which 



th^ retrieved, and ^at a major deteminant of tiie utility of the retrieved^ 
i^fo^mation^ irasj^^ errors made by toe student, ^if 

€OUld result from the fact that a high f regency of syntax errors should 
^ jaake -it more^ dif f iouLt Jor a novice user to - retrieve- any kind.- of inf omation, . 
let alone useful information. 



r 



td) Post-Straining coEsnents* Wien asked to give their posi* 



sojse fee&ack. Hany &f thm positive Mmments were really directed torard 
the positive aipects of on-line se^chin^ general, sudi m tiie ability . 
to e^and and^co^lnm sets. Overall^ thm assets of tlDh mcst appreciated 
by this group of users were the availability of the help Pode# error 
. .notations^ and the ability to make corraQtions ^before gontinuing. In = 



1 



addition, several* xmmmm noted that^Uia c^mands were well dsfinea, aid 

\ - ^ ^ . . ■ . 

ttat ^e ^Mtrmo^on were clear tod easily applied they were doing 

; ^teir.pira miMnstrained^^^^^f^^^ ass 1 stance Etode,; A few of the p^rtl- 

- elpanta reported ^JLOT^reel^ tKe ©p^rtimity to give input abDut ^jfeT ; 

i^- — - - ■ ■ ' ~ ■ " - - ■ "■ - m ■ - - ■ ': - - - - -— ■' ■ ----- ■ . — ■ 

ralevance af the it^as ratrieved* * ' \ . 

' #Mhen asked to give their negative mpreBsions of ItDh, sonie of the 

_ - ^ ^. ' A ■ . ^ _ - . 

\2sers TOmplained of too mudu infosMtion ^heing provided in a very short 
=^^^^€=S^i^e.-^We^~^^^^^ 

tion._ Although SOTO of 'the ^sers complained that -the help InstruQtions 

were gonf using, or too lengthy, other assets of toe instructions and 
--jgfoj^gg^gjji -pgQV i^ BB&MS'^o^^ iMVor ^ka tcKy to'^ the us qxbt~ - - 

exajaple, a few parjbieipants indicated, being -imelear about the '^hierardhy'* 



(4 ^ ^ ^pf ,, ^ff-ir^i^^r f-h^ :^ystem. and felt that more information 



would have bean helpful^ The single feature %rtiich brought the greatest 
iiunLyj. Q -t- s^omplaints wa s -tiie— f-IDfr-resp on se t i^e— lri»itattoi%y-i-«e'»^l^^f<-^ 
in ninety seconds if the' user ^|ps not respond* Anotiier source of irrita-^ 



tiof\ to toe user was toe machanieal delays atid disruptions of -the system' 
wfiiqh resulted fr^ such problems as network disruptions or plMiOG computet 
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problems. Finally/ the IIDA assistants ntoted toat many of toe participants 

- - * ' . w - * 

felt uncomf stable wito toe length of zeroise toree* Often the partici- 
pants reported negative responses to tWs ^ercise frrai fellow classmates 
iTOn-befo re ^ c^rienging— it— theMelv^s^^— X n f a cty^on e par tial' pa n t re fused 
to do ^ereise three on the basis of such reports. 

IfliM asked if toere was anything toat toey did not find wprking with 
IIDA tiiat" they felt should fee included, many usirs kequested a lift of 
^TO^nds, as they fou^d Continually looking back over toeir printout or 
referring to toe help library to b consuming. In addition, toe 



Kg£s^ suggested a r^^]^ af ^tailed luAds of idaaa abo&t vmrioys time 
saTlng TOtiiodEJ For ^as^la^ one participant suggested a table Ustijig 



/^irloi:^ mdf E of belp a^milible a pnTlsion for d^eet aooeas. 



other suggestions Inoludad auoh ideas as s s^thod to dfseon^^ee to^ 

KP^HO opEsiaiid* - Wi« a^ed to list sltiiatiosi in i<hiA %wA did tite 
wrong tiling ^ a n^ber of tile participants provided detailed fee^adk about 



previously uridetected progr^maing fla%i^, 



If the users felt tiiat they had learned, ^ough about seardiing that 
^ttey^ooi^d^ earch^on^ thait^i^ ^wlthoufc^JlD^. aseistanoe> .^tiifiy^wara.-asto3^ 



vh^ ttey first decided this* ~=Pewer tiian one^tiiira of the =users responded# 



md all*reported that they felt they had mastered searching eiUier after 
exercise threes or during the search dor^e in tiie assistMce mode ("exerQise 
^four") * When as3ced to ^plain their ^ «gon5 for feeling_^ ^at they co uld 
or could not search witSbut IID^ assistance^ those who felt Sey werfe 
f TOLllmr^OTOui^i-with 

assistance, Thoie who did not feel comfort^le with the ccramands/ or who 



felt the system was too cempiicatedf repor^d needing more experience with^ ^ 
IID^ before they could search on tiieir cwnk . , , * 

(3) Discussion . It seems clear from the foregoing information 



tli^t by the end of the suamer of^WT^, ^e IIDA systCTrwas in good enough 
shape to be able to provide the type of training and aisistMce necessary 
to enable novice users (at leait som novice users) to do on-line bUblio^ 



graphic searching. Thus^ M^^f the major goals of the project, had baaa 



ace^^llshed, i,e*^ the development of sudi'a system in a compu^tr has fed 
* 

interactive framework, it is also clear, however, that a nwiber of 



siAstanti™ ^m^l^ssB r^^ined in "that, while the average pereentage of 

* fttl ite^^ete^ved fee te^nlgal writing user groi^Jwis 601, there 

" i ' . " -- «■ 

ypr^ q^v^^at n^^rp wHq u&re dissatiBfied witti ttie system^ or v^o were : - 

mable (or possibly mtwilllng) to jfetrieve iuiy refer^wp %Aiieh they 'felt 



to be useful* Had, ttie evmluation of the IIDA seardi asslstMee system 
raded with the studies reported herei a reasonitole smMitioai o^&e results 
of the.projeet would have been/ "It seems to work,^ but fliajor improvefflents 



^CTaln to beTrads 

Jde sunme r § tudies wer # f however ^ pilo t tes t ing for s^ s e^ue field 
as^ertoents to be reported in the final Quarterly ^port* Several lessons 
^were^ iearaed^ f rOT^^S 

deiita# which led .to major benefits in the . field atudies* One obvious 



.^efi^ft^ff-TOi^-tiva-pfel.o4^ sto p^ly— ia-th a use of the instnjrtlnni^l 
and assistance pro^ranis with "real" uaers searohin| "real" problems* There 
"^wlrg^ome^RWrt^^ uiiw \ mi mir~tn add iVioKf^this- 



testing # along witii the network tranaaiseion and other technical probl^is 
encountered during the su^ir# niotivated a detailed search by search com- 
parison between search transcripts and the loachine kept record of 
search activity* TOe results of this comparison^ led to improvements in 
the operation of ^# automatic record keeping functions of ttie variotaa 
^programs * - ^ 



Ipse^^mdditlonal^OTefi^s^hiA-acerued^f^om 
involved tiie cjuestionnairaa and the reiearch asaiatants. Thm extensive 
fee^ack from _tiie. technick^^^ included cemditary about not 

only tiie aysten^ but also the questionnaires. This fee&ack, and the 



examination of the data provided by the t^estionnaires thCT^elvea, led to 

' - ---- - - - . - . ■ : - - ■■ . ^ ..... . ,\ . . .. . ■ . 

a ^nsider^li shortMlng and sorae'mi TOodiflcationff^in forinat and atyle = 
of the questions in the development ^of the field test questionnaires, Thm^ 



asaistiaits inTOlved in tie prajert gain^ inTOliatole ^©cperimf e from the i"* 
p^Qt tasting in daaliag with toe prabl^^ of wording witii the mystemg of 



woridAg^th the i^ers being teained, of eop^g witit the prctol^ of 
BGhedaliog ttsacs# o£ Mnf ©ming to toe ne^ida of- pla^- Bte 



ijisighte io;, toe operation of toe s^stCT to 

to ad^ttonr as a result of toe e^^erienoea wito the taehnioalxwriting 
^^M^ft^^ifjgatign s^MJtt^,^ ^^ 



eitoer clari^ the InfomatlOT or to sake toe exeielBe TOre palati&le^ Thm 



^ ^^pes of changes isade to eKeroise one ware prljoarily toose of clapification. 
____.For_e3i^sEl^/ ™.*tJirst^ soro# _s tM^ts _ .^ peg te^^^^ reteieye aotu^^ ajbout 
^ -their topio ratoer toan toe rafarmces* Others did not ajutially^s^^ 



' realize that the TOmmands in eKercise^one were td ^ leariled for futura 
T^e* ' Still otoers ^splayed coaGegtuaJ. msynders tiding. At toe tima of 
toe SOTaer testijig# axerelie one eniployed aa example saar^ on Library 



AutOTation as agjormat for introduoing toe various sear oh oonBnands and 
..L^^^^ A^e- At one point in the aear<^ r IIDA-askg^to e - fee r 

to mter/raPM^CLlBl^Y* A few of toa Mers, vara not mtiraly olear about 
~ ~KS^ to ~y^^ 

guently thought toay should enter EOTAND LIBRARY fto* matter what the tern 

was that was' to bfe' e3cpandad. All of toase pr^lams wara spotted early in 
— ' . . • % 

toa testing # ^d appropriate ehangas to exercise one were introduced* 

- -- i _ ^ 

There was aJ^o an ^tensive revision of toe^s^^icture of exeraise 

threa. In its original fomati.tois axeroise oonsistad of ^ight f rases # 



— wito «tanVlve ^^isc^mim ^& imtmilmA ^ l^b^ 

all of toi'material in ^eroiee toree^ toa total ^ine required to eomplete 
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^^©^axareise averaged ^oijt forty, minuter* ' siJbteqt^tly phMged* 



to be «jMedrrto only therinteMuotO|y tiie vario^ adyanoed 



training materials * the user is now req^uired to take a referOToe .walk 



tii£o\^h «eroise three to arder t© beeom^ f ami£iar with the contents of 



the adv^oed training material^ e*g* # ta. lav*, 6iat oha ^Bn tryneata with- 
out baJjig r^uirad t© aatually try a teimcatidn at toa, t toe ons^^ first 



leayms^ trmoatidn is possibla* ^ iser is gi^v^ tba op^on of lep^ming 
jaora in axaroisa ^tjttaa- 

returning to^fthi oontents o& (□eerclse threei ±n the fom of the help 



llirary/ *flien necessary "during work in tiie assistance njqtfe. 
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Search Niaabar 1: 

* ■ ^t/'' 

A user wants a search on aehool busing and raolal 
integration in alimentary tahools. 

ne would like about 60 eitatlons. 

Dae: t^C (file 1) t * 

Search Himbar 2: 

A user would like infarmation^ on collective 

bargaining tH college and university libraries < 

Re wants about 60 citations, 

Use: ERIC (file 1) ^ ^ . 
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APPENDIX B% Fuuidamentals of Library and Information Science User 
Gro^ Questionnaires 
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Btbllographte Scagchln^ 



Name: 

Code Word; 

Year of Coireges 
' Sex! ■ 



ladlvlduallEed Instruction for Data A^ctsi (IXDA) Laboratory 

105 Stratten Hall 
o , Susaaeri 1979 
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Wfe would like to have you give us some tnforciatton about yourielf so 
Chat we have a better idea of the ncads and interests of our users. On the 
following pages you will find several questions. Some call for a wrttten 
aniwer others, for a check mark indicating your choice among several alter* 
natives. Even if you find socie of the queitlons at range or inappropriate 
it is in^ortant that you complete them all. 

1. What is (was) your undergraduate oajor in collegel (Be as gpecifi& as 
possible) ' 



2, Year undergraduate degree received or eKpected, 



3. , bo you have any graduate degrees? Yesi 



Nos 



tf^y#^r^wh0t--4€gr^#<#J— Itj-^ — (#^iT^4W5^in-r#y€ho^logy)- 



4. Please give a brief description, with field and title, of &ny full-time 
jobs you may have held during the last five years* 







i 










Please give a 


brief discriptlonp with field and tlj^le, of the full«time 


Job you would 


moat like to be working, it five years from now. 


— ! - - 
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Hew much courst vDrk btyond the High School level have you had in each of 
the followlngi (please cheek one boK for each area) 



Moria 



^ Only 
Incroductory 



/ 



couraos 



Znt^dueCdry 

plus 
Inceraedlate 
courses 



Advanced 
Training 



Math 



Coi^uter Science 



Fhyaical Selence^ 



Engineering 



Biology 



Social Sciences 
(eVgrir His fcofyV"^ 
folitical Sciencei 
ete j 



Behavioral Sciencei 
(t*g* f Psychology, 
Sociology» etc.) 



Have you ever had anyofte do on-llnt cosyuteriEed bibliographic searching for 
yOM? ' ' 

Yesi Nai 



If yesf roughly how many times or how often? 

Have you, yousilf, ever done any on-line searching before? 

/ 

HontI Somei A loti 



Do you know any co^uter languages? Yess Noi 



If yeSf which one(s)? 
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How would you describe your coc^utar prpgratmlng ikilla? 



lNoh«€xistane 
iPoor 

iGood tnough to git by 
iQuitt good 
llKCtlltnt 



Have you ever used packagts of statistical programs such as SPSS ar SAS for 
data analysis or other itatlttiesl workr . ^ 



Yesi Kol 



If yes, how often? ' i Very rarely 

f J lOceaslonally 

^ ^ *gg£ii1flr1y 



iQulte often 
Ijery often ^ 

When you have identified a classroom problem, how do you identify the 
information you^ need. That Is, to what extent do you rely on tlie 'following 
Individuali or resoureesi ' ' 



Asking teachers I, 



not at all vefc^ft|tle sometimes a moderate amount considerably 
Asking frlenjii * ^' 

*^ ' W" ^ ^ ^ ^. 

not at illVl^ry Ifttl^ sometimes a moJerat 



e amount considerably' 



Askin^lparenti % 

Mm^, 

not fitv|i^V''^i^y i^^^ ^ moderate amount considerably -^r^-' 



not at^I;^S^W?^e;isbs.&ttmes a moderate aidount considerably 
Asking ^^ llBrfe^Sa)^i^v U ' 



not at all,*V^y'?^Wb"sMit^ti3 a moderate amount considerably 
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Intuition should play very littU role in solving probUtas; 



Strongly agree Agr^ Uncertain Di'sagi^ec. Strongly disagree 

\ 

I da not like using the computer for claSiroom assignments^ 
f . . . . \ 



Strongly agree A^ree Uncertain Disagree Stiongly disagree 

■ . . ' ' ' . \ 
I believe that every 'problem should have a clear solution: 



Strongly agree Agree Uncertain Disayree Strongly, disagree 

_ A ' 

One of the benefits of an education is that It helps you to 
learn how to solve problems: . \ 



Strongly agree Agree Uncertain. Disagree Strongly disagree 

There should be a set of rputlnre procedures to follow in 
aolvlng problems: \ 

V — ym 



complex 



painful 



\ ■ . 

..J^ . : unsuccessful 
worthless 

if 



frustrattngi 
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agree Agree 


Uncertain Dlsagr 


ee Strongly disagree ^^^1 


: using a cor 


\ 

nputer for a' claasroom assignment will be^ 






\ : 1 \: simple 


very quite 


slightly neither 


slightly quite ver^^ 




\ a 


\ 

\ I I rln teres f.ing 


very quite 


slightly neither 


slightly quite very \ 


« ■ I 

— V • ( 




» ,^ # : I pleasurable 


vary quite 


slightly neither 


slightly quite very \ 






^, • 
I % \ \ sucaessful 


very quite 


slightly neither 


slightly quite varjl \ 

6 \ • 


* * « 




: : 'valuafcle 


very quite 


slightly neither 


slightly quite very ^ \ 




• • 


• \ :nrft frustrating ^ 


very quite 


slightly neicher 

5 


slightly quite very 

. ^ 

\ 

\ 
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\ 
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' BtbllQRraphtg SearchlnR 



Po$ t *S ca ?eh Qui it t anna 1 re 



\ 



Code Word I 



Indlviduallztd Ins true tlan for Data Access (IIDA) libera tory 

105 Stratton Hall ^, 
' Su^er, 1979 



« 



ERIC-^- : :— SQ 
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■ ' ■ " — — ~— — 

How do you feel about tliG amaunt of effort you rnade during the siiArch? 



very dlRsatiified disiatlsfted satisfied very satisfied 
llpw do you foel about the amount of timo you spent in the ptocess? 



very dissatisfied dissatisfied satisfied very satisfied 
How do you fe^l abQut the kind of .asslstarics'^you. receiv^ed from the system? 



^ vory^ dissatisfied dissatisfied satisfied very satisfied 

How do you feel about the procedures which preceded the actual search? 



' very dissatisfied dissatisfied satisfied very satisfied 
How da you feel about your total search eKperlencef 



^ ^ very dissatisfied dlrsatisfied satisfied very aatlsfled 
HOW do^ydu feeI^abo^^the""tteins^^^y 



very dlasatialfed dissatisfied satisfied very satisfied 
How do you feel about the search as a whole? 



' totally useless useless useful eKtremely useful 

■* , . 

How do you feel about the results of the search Itself? 

I * * I ^ • f 

not at all useful moderately useful very useful--I, got what I needed>^^^\^ 

Hav^ you ever done any previous research on this topic? ^ 

Yesi Not * . , 

Do you know of Items on your search topic which were not retrieved? 

r N0| I know oi none and retrieved at least the nmber of items 

*I eKpected frOT the search. 

YaSj 1 knw of a few items which I felt would have_ been retrieved. 

^ Yes, I know of several items which I cKpected to retrieve. 
No, I know of none but I did cnpect to retrieve a few more items 
than I did, ' - - 
____ No, I kfiow of none but 1 did eKpect to retrieve many inore Items 
* than I did. 
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Ilsvc you ever done any previous research on chc topic o£ your sear||^? 

What percentage q£ the items retrieved would you say were: 

Very useful: ^ - ''^ • ' / 

Useful: 
Useless: . 



, . . T©tal: 100% . . ^ 

WQuld.ygu rccorrRiend this .$yit^ *to jfour frien YOi : No: 

The proceis of searching .was similar to what I had expected: 

: : : ^ i : 

strongly agree agree uncertain disagree strongly disAet^e 

The IID^ searcTilng just, completed was fenjoyable: * 



aS'^joyable: 



: : ' ^ : i \ : ^ *^ : 

strongly agrfee' agree uncertain disagree strongly disagree 

^ ' / ■ ' 

fte IIDA searching justr completed was. sat^ijfacto^i 



strongly mgree agree uneercaln disagree strongly disagree 

* 

The IIDA searching just completed was helpful in working on a class 
assignment or problcig: 

: : : ": t 

strongly agree agree uncertain disagreb strongly disagree 

Th6 IIDA searching just completed was instrumetital^'in working on a 
class assignment or problem: ! 

■ ' ' * . ''""^ ■ * 

: ■ : \_% : : % 

strongly agree ^ agree uncertain disagree strongly disagree 

The IIDA searching just eompleted was characterised by instructions that 
were easy to follow: 
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Strongly agree agree uncertain disagree strongly disagree 

» 

IIDA search system I Just ■worked with was frustrating to use:^ . 

■ f t : ' - . 

strongly agree agree uncertain disagree strongly disagree 

r " " " " " " " " . " - - . - ^ ----- ------- . - 

The IIDA search system I Just worked with was stimulating to use: 

t i v * ' ' 

itrongly agree agrcaN^tcertain dLsngree strongly dlsagrde ^ 



llow much eontrol do you feel j^ou Had over th^G results &d the^ search? 



none very lictle rnoderatc considerable cornplet^ . , ^ 

Hw much understanding of the information system do you feel you ^gained' 
!thfQugh your Gxperlcnce? ^ * ■ 



none vsry little moderate cansidcrnbie complete" 

How much cortdlnty is there that you would use the system agal-n If you 
had access to it? ' 



none very little moderate considerable complete 



Th6 use o£ IIDA was easier than I expected: 



- stronEly asree agrafe uncertain disagree strongly disagree 

I would «coninend. that others who want to learn to do bibUagtaphic 
searcfiing learn l^hrough ^IIDA: ' * ^ . 



- strongly agree agree uncertain disagree strongly disagree 
The assistance mode , of IIDA helped me *to iearch more effectively: 



strongly agree agree uncertain drsagree strongly disagree 

ypu feel you have learned enough about searching that you could^.ieatch 
on ydax^cm without IIDA? . ' ^ 



i 



If yes, when did you first decide that you knew enough abqut the 
process to do it on your own? 

* 

Please explain your reasons for why you feel you couid gr could not 
search on your own without IIDA assistance. 




please dGserlbo ypyr overall impressions of working with IIDA# ' / 

♦ . - ' ■' . 

* ' * '- " 

Positive Impressions^ C?hat did IIDA do or say that you liked most?) 



Negative Enpreiiionsi (What did IIDA do or say that you Hked least?) 



Was thora Anything that you didn't find In wotking with IIDA that you 
think should be ineludad? . - 



; When ypy live Identified a*clM^*«onw'ptobl fdentify the 

- 3^ inf orrft^ That is what extent do you. re^ on the following^ 



Askinj teachers • * 



not At all vary littlefJoniGtliEe^^ Moderate Miount eonsfdarably 



Asking fri^di i 



t c 



not at Xl^l "Very little aomttijnes a i moderate ambunt eensiderably 

L „ ^^...-^ 1 : ^ 



i^sking parents L 



not at all very litcle sometMes a moderatfi,,^Qun^ aonaiderably 
"WfnyTo^rtie*'IiHMr5^ 



notrat all very little SQmifiineV "a 
Asking ti\m librarian: = ^ 



nont oXIpveryTrrt'tTe somft'^mea a tfOTersra amounri^Qn^^ 



Using a .compyter far a clasfroom assignraent was 
compleKi ' ' ^ : 



I ' I ^ I I ^ ' . y : : simple 

very*quife slipWiy neither slightly quite very ♦ 



3or Trig I 



IhtereaCfng^ 



vary q^^tfe stightly neither slightly quite very 



painfull 



rpleasurable ^ 



un successful 



very quite sliglftly neither slightly quite very ' * 



viry quite slightly neiCh#% si^titiy quite very 




succesrful 



war til i 



^valuable 



^- ^ very q^iAte slightly neither sWghtTy quite very 



^. iyftfy quite slightly neither slightly quite^ ve^y 
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I dld^q^o^ like using Che &omput€r for m cIsssVsot mssl^mtnt* 



stronglj agfee agree une^rcau disagree scrangly disagree 



>There should be a:-1^£ 'ef jrbutlne proeedures follew ^ solvitig ^robleaB:^^- 



: stroiigly agree agre uncerts to disagree ^rgngly dissg^ i 
tataitlTO should, play" ve^ little role in solving ^obltas: ^ >^ 



Strongly agree agree uneercaln disagree scrongly disagree ^ 

tee oC the benafits of aA education is chat it help you tolpearn hpw ^ # 

-to^solva-problent-i™^^-— 



^ trongly-agree-agr ee - y noer^^ln -d i smgree^ s 



I.bnfi^leve^ that every problem should have a el€#j solfitloh^ 

■ .... fl*. '-^ 



strongljt' agree agree uncerffiln disagree strongly disagree 



>4 



4r t ; 
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. 4 



4, . 



v ' - ' ■ \ - ■ /■ f - . . ■ ■ ■ ^ ^ 

t - ^^plvlmg a i^searfcK several steps, some 

<^ _ - timeg bafore^the t^oblem is solved^ ' % 



; -^v -^"^ ^ ^^^^ % a) recognising the problem eKists 

, „ ~ bX def ining'the probl^ 

^ #' ; 'ji ^ ^ brewing the problaa up into 

^ ^ * . d) seleeting one of the sub^probleins 

" . ^ * -c) generating optiona 



^ * g) ii^lepanting the ^option 



f) ielecting an option 

g) ii^l^anting the ^opt 



- ^ --^-^^^^ (^)-ln ^he yrobl«*fqlying-^roeess-i^ 

, ®«erlals s^t useful ? * ^ . ^ 



At what >olnt(s) have you typically yaea bibliographic searehtng?, 



%^4At what poljit(a) can you Inmgine yDursell making use of the^e reioureeai 



iearehlngt , , - ^> " ^ . 



« 



III 

II 



,......i.......^.,.t. 
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bow you feel abouc the sMtMent. ^ * - ^ 

^trtry^ :CO"be the doQicaat per ^ : - : ^ 



usuElly oftan s^^timaa oeanalanally rarely> never 
1 try tQ talis charge ©f things when I ^ w^ith people # 





*; I 


try to have other people do thing i I want done« % 

m m • - • «' #_ a 


• 


% - 






ugually often simatimes eecailonalLy rarely never 








• I 


try to Influence strongly other people 'i actions* 

* • - • _ - * ^ * • . ? * 




h 


■ usually often iomttlmes ocftas tonally rarely never . 




I 


try to influence strongly other people's actions* 

















I tr to ha ve ot her peopli do t hings the wa y I want thmn done/ 



usually often sometimes occasionally rarely never 



I try to have other people do things the way I want them done. 



ffiOit -.people many people ssome people a few people one or two people nobody 
I take charge of things when I'm with people, / 



usuaj^ly often £om^timcs occas^nally rarely never 
r try to take charge of things whe l*m with people. 

- --i- .. ^.i.-. -i-- m - - -- - - - — T— "-T7-:--^ I- - :"Z":---."-r ; 















♦ X let odier people decide whmt to do. 






. . m - -..^'^ ■„ 1 ' • ■ * 1 " -*---'* If- 1 1 * --"^^ 

^^r rr^^ of ten sraetlmes occasionally rarely never 

.... i . , . . . .. -.^ ■ i . - . ^.r 






■ ' ^ - ■ .... ^ .. . ^ -. . ^ -. . 

1 lrt--^f^1vM--pRaplft-dee^ldMrfhat tQ-A&^ 






; ^ ©ost people laifiy people sra^ peopl»e # few people one 


or 


two people nobodi 


, / 1 let other people take charge of things • ' , V 






^^-f. ■ - - - - ■-- • • -^^■'^ - - % - • -■ - 






^ Sliest people many people s^e people a few people one 


or 


two %eople nobod] 


I let other people strongly Influence my act ions, 

• • » ■ , * 




• 


*^ most' people many people some people a few paople one 


tft two people nobody 


^ I let other people strongly influence actions. 


m 9 * ■ * * i m * 

* 3, usually often aamiCimes occasionally rarely never 


1 let other people control actions* 


. ^ usually often some times occasionally rarely never 


I ma easily led by people. *^ . 

# ■ ^* 

« • * ■ ..___*__ f ■ - 






usually often sometimes occasionally rarely never 






1 let people coijtrol my actions* 






* # • • ^ 






most penple miny people some people a few people one 


or 


two people nobgdj 


1 am easily led by people^ 






- - ■ . • ■ ^^—^^^^^ « ^^i™ ——————— , BB, « — — ■ 

_ • * • * -* 1 






most people many poop lo some people ai^few people one 


or 


two people no bod) 
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k writ^i for advanced studantm 

^i:^^^mi^mh^i^ smimiti^iM fislds# Xt is based on th# assisiptidn that^ 
vhra jQir beeam prdlaaiionals you will apand a conslderabla pai^t^of ywr 



All wrltjUgj|^nd espt^lall/' profssslonml witlng, rsiiilts^ from th# 
wpear's aesssmsQt o£ the rhetor tieal sltuatlea: the rel&tloashlp among 



to say p saying It (es^resslon) , and laytog It (forms). Tha lirst and seecmd 
^^r^hema^«a^tai^n^^^tB^;^o^e&;^ ^tha. cwrsa provi^da the^ p atttm^ot. 



organi^tion £or raadbgi and disousslons* Tha third, forms» is discussed 
itirou^igut' tha'^eoinfse at app^opriata pokits* - ^ 



TOITDiO AiSICmffiNTS 



-^--^^ «^-^r--" - -^-^ - - 

Jmgh ji y^5jt^^i^p_r a 
3ii^0Si ^a prop<^sa^ %n o 



sponsible for levan wrlttan assignaants: a sioto 
on BW^^B% ^a!*propQsa^ %n oral prograss raportr^a p©pula¥ article, irTat tar 
-of traftsfflitiift% ' aa ^^ri aatr- ^ ^ift^l report ^ — -- 



Attendance £n eiais is maOTiTtory. ^xcf ssTvF^naScWe^^ 
rasult in failure of the coursa. 

. ^ .... ..-..fiRAOlNa'— - ' ■ — ---^^-^ 



Final grades will reflect the student's pcogttss over the term in 
^writing clearly and effectively and in the appropRate forms. 



Assigmants ara due on tha datas indicated on.tha syllabui and 
InstiiJCtions for Assigr^ants* Lata papars will be raduead in grada# 

Cll^M. EEQUmaENTS FOR FAFERS 

1* Typa all papers 

2« Use b 1/2 by 11 inch paper 

3* Leave I 1/2 Ineh margins on all aides 

Am Double spaca unless spaciflcally instruetad to do dtharwise 

-binder — — — - - — - — = 



6m Submit aach with a covar shaat stating your nma, ni&a of aiiignmenti 
datei faistructor 's name ' ^ — . — — ^ — ^ — ^ 
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g»tence) % -■ ^ • . - 

- per^li^n^-^tb. ji^-l'laid? G^s full eltatlcms tad m brlaf deserlptiea 
2. tfhld £l^^fj^^^3^;^e7j^mslda^^^ significant in you^ field? 
miffis t€t fU^.-Z-'l^a^:^^ tbe artl^lss but dssc^ibe Aeir 



g^^Bl:^^%^3^/MStfi Mffm. and style of prasentatiOQ. 
^37™iftScff Ss^^MiTijpptt^ of thaTnstruetlehs t^^Authors 

paga(8)^£i^'^V0Ii^ q£ tliii^ jQu and/or nota cha npmas of ttia 

s^la, gdUeV astist authors In prapartog sanuscripts for 



2-3 pp, 



HtOfOfflBj-COptionrfA-ite^; 



Attdleiiee: a poCtiatlal ' sponsor • - 



* ' m^u» IndusttMl Vsu^'rvif br^ rartMOh "^Isiyv, ata*).. It should ba tail^rad 
/ 'to«|h* #pbnwr!*'sp#e^l£iM^^ It sh^a^^luda fiiMd^^^ wifhin tha taist 

J '^a^y jtejgljijtt^-^ are ^alavi^lK^Ita iM^a^^^^ tp P«wada__^ 

^/ tbE spons^f^o^alistf "you tac^^dertSa m ac^^^ l«3J^^to*tfia ftaalTreport. 
In ganarml^- ^Jfop^al should Includa the f^iloif^gi'H^^.^ - 



2* fta^Kgi^ou^d. ^Establish the eontait for the tavasti^tio^^ glva a 
)iis^tty\of the ptoblm, the naad fo^^^^solutlcm banaf its 

ifiU z^^f free your invastigati««jj ^ / ,^ - . V- 
3^ :^Biiftir'P^o^t^ur^ you will usa to c«^l#te yo«s"to%^tigatlon». ; - 
4* ' Prl<Vio^s^e3^paTfeaca and faellitias dascrlp^ion.' %> * ^\ - " \ x 
5. Es£&iat# ^f likelihood of sueeass. - . C V'^, 

4m Li»fc ^f^ f^^^^ pagfomed and task sc%m&^m\^\ /,t « \. v,uV 

7. fcei|UMt .for paralsslon to undartaka work*" 7//" • . "^^ ^if ^ i, 



1^ lOpt ion. B^- toa proposal should ba is tha fora b£ a lettafe aWMjfsad,^ ^.'V'f "t: 
^i|itg^;, pf, . a , partiaulsg journal that might ba Intarasted In a ffariay"jgf^ , .. ^ 
* l^teratare o^ a Jlopio of interest to you. You should find qu^' the tt^'^ Md ^^^^^^^^^ 
addrasW of the editor of tha partieular Journal you^thtok mQS£.si^tta^l^>.%...|w.u,^'L 
.Address thii following topics in your letter: ^ 



f. 1 , 



^ifp^ ' to examine is sigcilflMnt to reJuters a£ the journal at ti^« 

Teatative plto o£ davelopmant of your 

you wilV or vill not ^isou ^ ^ = . „ . 

3. Survey of the litirature. Show your f as! lift Ity with major artieleSf 



dlscussiom you, wiliL present in th% ^rtlole^ 
^* Tour qua! in^tiott^ to the editor am soffieone 

suitable ^ to do the rilview. . 
$w Sehedule« ^dlcate when you could iutimit tfte artiole* 
S« Final sales pltchi Mswer the editor's legitimate questleni "Why 

shou ld y ou do an article on ^1^ topi o fo r readers of this^our^l 



r ' J 3»4 pp. 

Audience I your islas^iates 

The progress report should be addressed to the class. Its pur^i# is-to 
report ~p« 

-or4gin#l-pgop osa l - i n-4e€€r4bta8-yrogr^##-mad^i— V ^ 
sodificati^s in plaiii and the current status of the work* 

POP UUOt ARTICUe ' ^ 



Duei ' ; * . , 

Audience: the^; readers of a nontechnical magazine or newspaper 



The popular report is intenfled for a popular audiencei specifically readers 
of a magazine like Popular Mechanics > Smithsonian a Fsycholo^y Today , or - 
Today's Health * The report should adapt information gathered for ^^ur 
"f inal report ''for this audlence"*^^ 

viewpoint^ level of language, organlEatiani style, type of material of 
interest, interest^getting devices such as titles and leads* l{atch your 
approach to the audience; sell your report* You may inform, eoi^incei 
advise I or .entertain the reader* Your article should be lively and ' 
readable* You may be more personal than In your final "report* Hake 
good use of such intereat-getting devices as comparisibns, analogies, and 
^^crap les^hmt^le^ii^i^ h=^he^Kper^TOce^ 

the presentation. Catch their attention* ^ 



LITTER OF raANSMITTAL 



pp. 



Dui : ^ 
-Au4ienca4— tl 



fhm letter of tranOTittal should be addressed to the sponsor (Option A) 



o^^tim ^Ltm XOv^^^^) recelvs your flaml wmpoTt/ It should 

ia ^ynm BO^ ifeik y<w«^^^^^wt late of fsr- to subait — 

_ yo%u# iA\restwll^g for yow final rtpo^^^^ ^ 
As abstraee It a sussi^. of the eontmt% of your final' report*^ should 



tha nsaue^^c^ort* Tha vord ^S^WT should ha tj^e^;at tha top ewtar 
of tha pagi'^ aapltal lettars« suma^ which" follows should be mltti& 

;i ;^ :u Itt sastw^a'^siid- (Sea PsycholQ^CTl Abstraot s f or^i^^ as . > : 



FIKU. RIPORT (Options A and B) 



Due- 

Audtaiiaa t sponsor 



The final TSport Ir-a raoerd of your-invasft|gation into a problem for tha 

audlanee's raquirmantsi you naad to knoy w^t that audlanca wants wd what 
aotion it can taka sftar raading^ your report. The results of 'your raseareh 
ghouldi b4 presented^ in an arganlEed and COTCilia jaanaar. yiiuals should be - 
toaXudad In places whera they will olarify po^ts ttada in tbe taxt. fha - 
report should eontabi an Int^duotion, body ud^eoneluiipn* 

; , 7-8 pp. 



Option B should ba written for tha aditor to wlHm y&ur^ proposal was addrassad, 
The raview of literatura sKould ^^a. a o^ s^thasls of what has Been ^ 

j^rittm JTO^aLtopift 

sparingly if at all. The artiala i^^^^ the form of other articlas in 

the journal In siattars of botH organlutlon and doeumantation« Enough 
evldenca dr^wn from current sources ^opld ba given to provide oonCaKt for 
your interpratations snd to support your ganar a lining statenients. 

. 7-8 pp. 




^ IM toCT^phte ^ Search ft a jj^op y - 



''Li 



Code Word ; ' ^ " ' " 



Collage 



-tear4o£-€ol l«ge i 



■V 



IKd^iy^duallsed' Ins true t ion for Data Acaess (IIDA) Laboratary 

J 165 Strat tan Hall ^ 

* - • ■ 
-^SwnmeTf 1979 

^ ^ t - - < ■ ' ■ s... .... 
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SlBce yai are part^of therfirBt scQu^*tp hmvi aoy e^teaslva eKperlfaice - 
vith X^A\ve would iika'to hava you give us mmm iaiotmAtion Bhots^t ypi^^mii?^ 



the foll^iiig pages you will find #sviyal questions. Some call for a w^ittari'C^^ 
InsMr others, for a qheak mark indicating your cholee arong several alteriiafei^ii* 
>Evon^f y&ii f Ind^^ it Ir; : ' V ^ 

^,„„iaportmt:^that.-yoi^feoiBpl^ ' .^^^ _.^.^..l^.^^^.v^^^^ 



1. What if (was) ^your undtrgratffiiia major in eollegel (Be af specific as 
possible) ' v ^ ' ^ ' / ' 



2. Year undergraduata degree received or eMpeeted* 



3» Do you have any graduate degrees? Yeai^ No: 

;"^ei71rilia tnaepf eeX 



it: 



4* Please give a brief desetlptlonp with 'field and title, of any full-tijQe 
^oh's^iTOn'^ raop^^W^ W^dut liig the last f Iv^T^eart — - 



5* Please give a brief description, with field and title, of the full*ttae 
job you would Qost like to be ^^orklng at five years from now# 

■ s ' ■ 

- — — — — - — - - - ■ — ^ -- - - .. ^ — " > 

_ ^ 
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' Hq|f «teh 'eottirse we Hlgh'^Sehool level have you had, in each of \ 

tke folloving^ (please cheek one hox for each mfiea} ■ ' ' - ' ^ ,| 



• iBtroduetoiy ^ ;\ 

Booe : : Oaly * plus ^ Advlmeea ; 

Xn(rpdyctory Incermediate ' Training 

coOTsis * , courses ^ * . ; . 



# Coi^u ter Sc lence % 




1 

1 










J* 




i 
1 


* 








I flryjitcal Science 








,. 


















* * - 


I 


, Biology " * 
















'Social Setenees ' 
















. Political Science 1 




f 














Behavioral Sciences; 
: ^ etc .) - 1 

















Jtoya-^you ever 



Yes I 



No: 



If ye§» Mughly how many times or Kow oiften? 



None: ^ BOTes^ik^ L_' A^lot: 



r er 1 angtrnge s ?^ ye§ : 



^H04 
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>. . . < )l.. - ..<.. ' N rt l. < . 



Ho^^would you describe your camputcr pragranining skills? 



■ i^ood enough tp^ get by 
_'*iQulte good ^ 

iEKOellent . . 



Have you ever^usad library packages sueb as SPSS or SAS £ dr^ data analystE 
\or other itatlstleal wor^? , 

\ " ^ea: ' % Nor ^ ^ ^ ' ' ^ ' -^^ 

■\ If yaSi how often? :Very rarely 

( , ;Qe€asionally - ; 

" * ' ^ , iRaCTl arlv^ 

, H ^ iQulte often ^ 

■ . " ;Very often 



When- you hava tffantified a alasaroom problem^ how do you identify the 
information you need. That ii^ tj .what eKt ant do yoQ rely on the following 
individuals or raaouraas? , . 



Asking taaohir B : 



not at all very little some^timas a moderate amounts eonsidarably 
Asking friends: 



not aC all. very little sometimes a moderate amount coniidarably 

Asking parents I _ ' , . ' > 

I ' t ' * * • 

not alp all' vary littla sometimes a moderata amount ^aonsldlfi.bly 

Going to th% library I 

* • • • • m 



not at all very little soraetimes a moderate amount eonsiderably ^ 
Asking the librarlani • * 



not at all va^y little sometimes a moderata amount oonsiderably 



i.^' Intuition should play very little role fti solving problems! 

Strongly afree Agre^ Unaer tain Diiagree Strongly diaagrea 
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»^ I^do not like using the eomputir .for elasiroom aasignmenta 

- ' ; " U I I s A 

Spyorigly agr|e Agree Uncer4taln Disagree Strongly disagree, 
. " ■ • / ' ^ - ,^ 

# .1 bollave.thalE every problM should have a.elfear solutioni 

strongly agree Agree Uneertain Bliagree Strongly disagree ^ 

m - , ^ - ^ ^ 

' O ne-^f-t he banelitg of a aducatton .1^ t h at lyh B^ lps y g^^ o . 
learn how to solve problems i ^ ^ / 



i 



-Strongly agree' Agree Uncertain Dis&gree Strongly disagree 

ft- ^ * ■ ^ . %m * -V ^ 

Th^^ should be a set ol Eofttne procedures to fgllow in 

solving problems: ^ \ . . ^ ^' ' 



- * 



Strongly agree Agree ^^ncertain Dlsayee'^S trolly ^fElsagree '\ 
I believe that using a OOTiputer lor a claslr^^ assignment^ill be^ 



complex: : . . i ^ - i J t ... : • simple 

very quite slightly neither slightly quite very 

boring: : : . r. ^ : .r- ^ i '' . r^^ : Interesting 

very quite slightly neither slightly quite very 

painful: _:_ f . : : r i \ ^pleasurable 

^ very quite; slightly neither slightly quitfi very ^ 

unsueeessful : ; i s • ; i ; s successful 

very quite slightly neither slightly quite very 

worthless; : ; i {valuable 

very^uite sightly heither slightly quite very 

<s '^v- ^ ■ \ ■ \ 

^frustrating! ^ : m • " i ; ; ^-r ; _!nof frustrating 

very^uiee' slightly neither slightly quite very 
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B lb 1 loRraph ie Search Tf a In Ing • 



Name: ... . 



Code Word* ^ 



© 0 



4'i 



Ifj^vldualiEid Instruction for P^ta A^fcs^ (IIDA) ^Labordfe|y 
^ : , Strain «all ^. ^ ^ 

BmmBtp 109 \ ' ^ 



f 
#4 



it 



,...,-.-jL:^ -44 ; 
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How do you feel ^bout the amount of effort you made during the iearch*? 



very dlsiatisfied dissatisfied satisfied very satisfied 
How do you feel about the amount of time you spent in the procesa? 



very dissatisfied dissatisfied sattified very satisfied 
How do you feel ibout the kind of assistance you received from the system? 



very dtssatisfied dlssatisfted satisfied very satisfied 
^ How do y mi-^^l^bout-th€Hpg«#dur&g-whi€h^ seare h?— 



very, dissatisfied dissatisfied satisfied very satisfied 
How do you feel about your total search eKperlence? 



^ yeT_y dtssatisfied di^^^^ vety satiffied 

... * 
How do you feel about the items you retrieved? 



very dissatisffed dissatisfied satisfied very iatisf led 
How do you feel about the search as a whole?' ^ 



totally useless useless useful extremely useful 
Ho^^ do you £ee| abo\2t the results of the search itself? ^ 



not 'at all useful moderately useful very useful--I, got what I needed 
Have you ever done any previous rasearc^ on this topic? 
N . Yes: _ _ Noi ^ * ^ 



Do yos know of Items on your search/topic which were not retrieved? 

" NOj 1 know of ffibne and retrieved; at least the number of items 
/ * ^ I expected froin the search, 

^ — Y ea, I >knQw^£^a^few-^t€ms.^which^I^faLt-wou 

Yes, I know of several items which 1 eKpected to retrieve, 
- No, I know of none but I did* expect to retrieve a few^^e items 
' ' ^ than I did, - ' \ ^ ' 

* No» I know of nonm but I did expect to^ retrieve many mSre items 

^ ^^than^I did. . ■ 
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-^Have you ever done any previous research on t^e topic of your search? 



Yefii_ NO! 



What .percentage of the Items retrieved would you say were: 

Very useful i 

Useful: 

Uselass: , 



' ^ ' - Total I lOOr ' , , • 

Would you recomend this system to your friends? Yes* • No: 



The process of searching was sljnilar to what I had eKpected: 

— — — - - - " - ^ -- — — — ^ — - ^ 



strongly agree agree uncertain disagree strongly disagree 
The IIDA searching just coripleted was enjoyablei 



strongly a'gree agree' uncertain disagree strongly disagree 



The IWk searching Just completed was satisfactory: 



strongly agree agree uncertain disagree strongly disagree 

The IIDA searching just completed was helpful in working on a class 
assignment or problem: 



strongly agree agree uncertain disagree strongly disagree 

The IIDA searching just completed was tnstriOTental in working on a 
class assignment or problem- ' ' 



strongly, agree agree uncertain disagree strongly disagree 

The IIDA searching just completed was charaeceriEed by instructions that 
were easy to followi 



strongly agree agree uncertain disagree Jstrongly disagree 
^The I^A search system I jusC^&rked with was fruslrating to use: 



atrongly agree agree uncertain disagree strongly disagree - ^ 

¥ 

The IIDA search system,! just worked with was stimulating to use: 



. . . ^ '-- - - * 

^ Strongly agrW agree uncertain disagree strongly disagree ^ 



69 ■ A 



Please dgieribe your overall impresslSIs of working vtth IIDA* 

Positive topresaions? (What did IIDA do or say that you liked most?) 



Negative Zmpresiionsi (What^did IMA do or say that you liked least?) 



Flaaae deseribe any aituations where IIDA did the wrong thing. 



"V ' / , . 

Was there anything that you dildn't find in working with IIDA that you 
think should be included? i 
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* i • 

How do you feel about the amount of effort you madi 



the seiVeh? 



( K 




, very dissatisfied :,4tis^^ sa^^^d ^> ^ very iatisfied 

Hoi/ do you feel about the amount of tiine you ap|^^fti ^dth^ prDcessI ^ 



VMy dissatisfied dissatisfied ^^i^f^ very satisfied 
How do you feel ab'but the kind of assistanGe ^o^^eeeived from the system*^ 



very diisatiified dissatisfied" 4atSJ 



led. very- satisfied 



\ " ; ■ 4o you feel about the proeedurei which ^ed^Sd >jRhe a^fifiS 



very dissatisfied ^ disdatlffie 



^^How dp you 4^el^ about your toftal searth exp 




S:. 



v^^^d t^sa t Isf tetf- dl i sa ti$f led 



fted\4*fViry satisfied 



ery iattsf ied 




.SMi^4€h you feel about the Items you retrievld? ^^h^ 



/'/"^ Very dlssatislfed dlssati$£l|(l^^^tlsf led very satisfied 



. i" - ' you 'feel about the search as* 'a* 

- . " - * ^ 




totally usele|s uielesM -:use£ul. . eHtremely useful 



J"^ Mow do y^u feel ab#t the i^iulta ofcw£he search itself? 



^'.no^ at\all useful ,, ■'m6|erately useful very useful-.* t. got what I needed 



ydu ^tvtfef jrdone any pre'ya.ous researc^tta this topic? 



r ^ ."4 



Yas 



1^ 



iDo you fcnow ^ items on your: sear^^opie which' were^ not retrieved? 



.4 ".w- 



_ NO^j I know of none and retrieved at least the ,number of items 
^ j ]j I expee ted f rotrf the-: search. - ! 



"YeSj I know of a few Itemi which 1 felt wou,ld have been retrieved. 
Ifes* I know of several items whlcH I expected to retrieve. 



AV.^^^^_^ii^Oj I know of none but 1 did' e^edt to y;etrleve a few more items 

than I did. " 

fVOi 1 know of none but I did expect to retrieve many more items 

~ - - than^ I did. -^^ - - ^ t . : , . . 
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Have you eveif done any previous tcssateh on the topic of your soaroh? 

Yesj. Nos 

What percentage ol the items retrteved would you gay were: 

Very usuful^ ^ 

Uscfuli ^ \ i. 

Uieleasi ' 



* Total- 100% 

Would you reeosimend this syst^ to your friendi? Yes: , No^ 

The proaoss of searehlng waa similar to what I had expected: 



strongly agrae agree uncertain disagree atrengly disagree 
The IIDA searching Just completed was enjoyable • 



strongly agree agree uneertain disagree strongly disagree 



The IQA searching just completed was satisfactory: 



strongly agree agree uncertain disagree strongly disagree 

The IIDA searching just COTpleted was helpfuL in working on a class 
assigraient or problem: ^ .i 



strongly agree agree uncertain dliagrec strongly disagree , 

The IWk searehing Just completed was instrimental in working on a 
class assignment or problemi 



strongly agree agree uncertain disagree strongly disagree 

The IIDA searching Just completed was characterised by instructions that 
were easy to follow: , ^ 



strongly agree agree uncertain disagree strongly disagree 
The XIDA search syit^ri I Just worked with was frustrating to uiei 



strongly agree agree uncertain disagree strongly disagree 
The IIDA search iysteta I just worked witfi was stimulating to use: 



O strongly agree agree uncertain disagree strongly disagree 
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H©w much control do you feel you had over the cesults of the search? 



none very little moderate considerable complete 

How much underitanding of the infonnatlon syitem do you feel you gained 
through your experieneel • 



none very little moderate considerable complece 

How much certainty is there that you would use the system again if you 
had access to it? 



none very Little moderate considerable complete 



The use of IIDA was easier than I expected; 



strongly agree agree uncertain disagree strongly disagree 

I would recotmiend that others who want to learn to do bibliographic 
saarching leara- through 111^:^=.- ^^^^^ ^ ...^ _ 



itrongly agree agree uncertain disagree strongly disagree 
The assistance mode of IIDA helped me to search more effectively: 



strongly agree agree uncertain disagree strongly disagree 



Do you feel you have learned enough about searching that you could search 
on your own without IIDA? • ^ ^ 

Yes: Kot 



If yes, when did you first decide that you knew enough mbout the 
process to do it on your own? 

Please e^cplain your reasons for why you feDl you could or could not 
search on your own without IIDA assistance* 
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Floas^ deserlbe your ©verall impresstona of working with IIDA* 

P©iitive topressionii (What did IIDA do or say that you liked most?) 




Please describa any iit^ations whare IIDA did tha wrong thing. 



Was there anything that you didn't find in working witTi IIDA that you 
think ahould be included? 
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When you have Idenciftad a classroom problem, how do you "^enttfy the 
information you need. That is, to what extent do you rely oo the following 
individuals or resources? 

t 

Asking teaQhers^ 



sot at all very little iotnecimes a moderate amount considerably ^ 
- Askihg-friendsi 



not at all v^ry Itttle sometmes a^moderate amount coniiderably 



Asking parents: 



not at all' very little iometimes a moderate amount eonslderably • 
Going to the library: / ' * 



I not at all very little sometiioes a moderate ammint considerably 
Askihg th"i ^ribrar Ian ; 



not at all very little aometmea a moderate^ amount aonaiderably - 

Using a eomputer for a elaasroosi assignment wair . 

complexi t t : ' : ij ^staple 

V^#ry quite slightly neither sligntly quitA^ery 

bor InE^ : i i : i : interesting . 

very quite slijhtly neither slightly quite very ^ 
' , ' ^" 

painfuli^ ; i ; i 1 i ipleasurable 

' very quite slightly neither slightly quite very / 

. unsueeesaful! - : r i z i isuceassful 

m very quite slightly neither slightly quite very ^ 

• * • I t I ^-^^aluable 



worthless : 



^ very quite slightly neither slightly qu^pe^ery 



frustratin g : > i ; ! \ y tnot frustrating 

A |Fery ^qulte s 1 igh t ly neither slightly quite very 




So 
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I did not like using thm computer for a classroora assignment- 



^ t 



strongly agree agree uncertain dlsagrM strongly disagree. 
Thare should be a att of routine proeedufe fo follow in solving prdblemsi ^ 



.Strongly agree agree uncertain disagree atron 



^giy;di 



iagree 



Intuition should play vary little role in solving prpbl^ai 



strongly agree agree uneerealn^QiSagrfe^^ 

One oi the benefits of an eduMtlon is that it helps you to learn how 
\ to solve probli 



strongly agree agree uneertain disagree icrongly disagree 



I believe that every problem should have m clear solution: 



strongly agree agree uncertain disagree strfongly. disagree 



1 , 
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Solving a research ©r design problem often requires sevfertl steps, some 
of whleh nay be repeated scviral ti^es before the problem Is iolved. 



Theie typ ieally involve pk*: 



a) rceogniiing the problem exists 

b) defining the protlem^ 

^c) breaking the problem up into 
) sub*-problMS 

d) seAecting one of the sub-problems 

for "solution" 

e) gentrating op.pions 
£y selecting an option 

g) ImplTOenting the option^ ^ 

h) ^ evaluation of the Result . 



it what point Cs) in tBe 
Biaterials most useful? 



"pr ohT^^s^tvtng^proe^^^ 



At what point (i) have you typ ieally ui^ed bibliographic searching? 



At what ^ point (3> ean you. Ipi^^litr^y^ making use of these resources' 



At what point(a) would ypu recormend to others that they try bibliographi 
searehing? - 



. ^ . ^ " Yqy Just eyylet^d a lea^ch ©n a tople of your oyi chogstng. ;* * 

:: ^ SopefuHy -^ous sear^ and its TresulCs will be useful in dealM^ wittea 
f w rmspargh project or alsigsiay whars b^lbiioggaphlc tnformatlQa is ^ #^ ^ 



. For &e questions oa^ the n^tt two pages we wuld* Ilka yoyf to asfume 



iffiat^Hit siteroft^ eatsplacW fia^ not yet bem dona at all* Rather; 

gfrfl^ffne tb^^ y^ti hAve Jf jsftke&te taplii and yo'u wuld .^llJia^^MMl ^dforM tioilj_.: 
about^that t<ipicv You aM going t(» uie^the. sarvlces of a libraiy 



iatenaadlary who will parfam a* jearah for yeint Tha following quest tons 
ask you : to .daaeyiba the topic of the search and to provlda other , 



TSfSrStton^wSrehnffia^^ — 
(Note: Sometlmai tha^ search results you artd up with are not the same as 



those you expactad whan you began iftie search. We would 4 ike you to ^ * 
dascribe' the topic for the^^^Mh you actually did rathar than the topic 



for the search you started ouiC to do.) 

— 



1 




79: 



SeMC^^^es t ion : 



i V^Flamse giVf lit yoot/ the problmiis 

- - to be isarched. Be spa^f^i d^&f p^^€s with special meaning ^ ^ 
Apptmd^s. list to your nait^a<iva 'any s^Qnyms/ cloiely*]rrtmted phfaseSj 



"confetti 



^^^hmve^ special use ^hat you wish tg. akelu^f' Use seleiicif i$ ahd^f < 
; > tectettfAl^as iiell as e ^aao it vocabulary,^ - ^ ^ ^ 



4 



2» Unless already stated^please^ Indicate any models » end usaSp or applioationi 
that wouldtbe Keiipful in ratrlaving useful^efera^ces far your problOTl . 



J ^ 
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3* Pleas% stmte My toply related co (or applications @£, or tlewi o£» or 
approacfams to your specific probla that are jgt af intereit if you wiih 
L^:_r:: jto :^BclBda:^ea:^ri^ eitgttgri| to my. doc^intsi^pn sy ch t^ics. . :^ . i . 



§• *.Plaasa list two or tbrse of the most taportant authors (and/or orgaalEations) 
puHllshing on your tdplcj complete nasiis, if knwn, are helpful. Please 



or organ izat ion i becaufs of prior fmiliarity with their publications. 



€|^«) you want searched: ^ 



written in a ©artiaulac lan^uaM^ - D oes nrt^Pttfr Retrieve Engl Ish 

only 

Retrieve onlv^ in Do not retrieve in : 



8« pb-you wiih to limit the search to a ^rticulariiliae span 



doea not^matter retrieve* 1977 to date retrieve before 1977 

Other 4 " ' . — ^ - ^ - 



""f. Please lilt the complete citations of two to three of the most useful 
articles on your seawh topic* / 







* 
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Cbe^k the word or grwp of i^rds foilowliig eash statercnt that best describei 

1 tty to be ths domlnEat peraaQ When I mm with people « _ * ' 

- - - usual^^ften s^Kti^s oceaslMslIy rarely never ^ ^ , . „ 
* % try tg taka charge thlnga vHen I w with people « 



I^St people Mny peopIe^ioiBe people a few pebple oni or two peopTe noSoiy 



1 -tw^to hsve-other peopl do th^^&-I-^ant done* — 

^^..^ jualiy^itaa^^OTetima^.^ ocMslo 
I try to Influt^ee strongly other people- s actions*; — 

* » ? " *) • • * 



usual l^D?tenVsometmes oeMsionSjfcy rarely never 
I try ^ influenee sAongly oth^r^eaple- s actloni* 

Elost people many people iome^ people ^ few pebple one or two people nobody* 



I try to fiave oth^ ^ :^ 

_ai£imLly^ of tea^ iQEaetlmeBboEMsionallx,^arely„ f: ^ .1 „ 

I try to have other pMjle -do things the way I want them done. \^ , T 

y./lr^A .-• *.- ''-.i-r * *^-.-- - — ' ' - -m - ' JL ' ' * - -"- - * 

moft people many people some people a £k\t peo^e one of* two people nobody 

' ? ^ * ^ . . . * . g .... . - 



« * * 



• - yfually often s^%t^es occasionally farely never . ^ 
T try td t-nkff e^iargg'qLf ttflhgg^ when I'm with pgQj^e- ' ^ 



usually*o£ten .s^etunes pcc'agiohAlly rarely never 



* I iftt dee^de what to do. 



iUet otiber people 3ae ide %rtiar *o i^^^ " 




I oth« peopU take ctarge o£ thtofs. ' " i* ' • , 

^^~*aiDSt people many people^ispoe people;a:^^s6pr^o^^ 



-1- let-ether-pwi« Strong ly-inftucnca^_act^Qns^^^..^^__r ' — -~ ^ — — 



usually often smetlaes occastoi^liy ra^ly #ivtr 



y .9* 

1 let oth^r people eottrol toy aef ions* 



usually of teh gc^tiaes oceasi©iially raMly Sivir 



I «~Batlfy" 1^ by^^opl^^^^ 



usAll^ oftir^ som|timai QeMSiorially rataly neveir 
I l#t paople contMl ssy ae£lans. 



U somf ptopla a f peopls Qm ot two paopU nobody 



most people many p^p 
I am emsilly led by psopfe* 



r:|::^-l » ^Tr-^-T-T -t-::^—^-^ j^^i^ 



'laost people many plop^^a lOTfe paople a few people oni'ot two paople nobody 
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' ^ the: follSfg^^ 
.5 pmaimh in Ea^elsg 4^ 



l.^thddB in deiigDlng buildings for eallapsa reslstuee^ 
.2« Froe^di^etf in aergtney core cooljUig la a auelMr r^ketor* 
3» M^iiMsiforr ths dasl^ JL drlllsd ^lar foisM^tlm (Mlgggn), 



5* isaljrsls of the bre^doim ehArmcMrlsties of diffuse 
— --—g^AcoaduetOf^Jimatl^ — 



6, itethods of d^lag elothas or ttntlle materials* 



7* 


Use of orbiting satellites 


as roote envlroimwtal monitoring 




stations* 






Use of wetlands ow marshes 


as waste^^^^ossl sites* 



9« ttm desl^ of sewage pluts' fUtration capacities based 
on the e^^eeted inf low# f ^ 









..... . .... - ■ ■ -.^ - ^ . ^ 

% \ tt; 






■ ■« . . 






'f/\ ■ ■ • 

f _ 






^./ .f • 


^ : - ' - - ■- - --- ■ i - ■ 

m 
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WX^NDIX Gt Vroee^W£&s for IterScorin? Oiagnostic Records 



I 




3 



■ ! 



,9^ 



i i .. i ^H lU- «4.-t i: iM| i taL , tj 



te a result ©f Pfflislderabie, on-liae iaterMc^qie, during -aie seardxes,. . ; ; 

sooe^oi iie"Wiitsn^^ were not «^a br^ -fterr-»^^j ~^^ 



1" " . ^ tta slar^ ^fiBMlfeta j#«|e M-«emdned by tiie* 

' y'' . IIBR staffs and the reported fra^mcy oft va^lou* fMiIts adjusted to 



witt' the systCT* ■ • • ^ 

^'^77* iii'iSst freiiSit ly "oe^ 

■£r^ noise on-rine, or from tel^honejpimetftlon Jisorders^ 
flerenee bajiBally «onsis4 of a string of characters and/or symbols which 
~^^ri iSedlipiy Toll^In^^^^^ 

by tlfe'user^ These glitches can be carieallad Jriot' to entering a coimand, 
but, be'ing unexpected, are often overlbol»dT mis is particularly true 



for the novice laser. T^ese nonsense chaeacters, which in aost cases can 
ib*:..««sily^disc%iaa±edj 
co^&d, which theg leato to an erroe, * * • 

Another set ^ of ^ehri±c*l rather than - syntM£ - errors-^ arose- fr^ th^ '.. _ 

operation of the IIDA systCT. Biese were 'not only deleted from liie syntax 
error categery, but also brought to the attention of the syst^ progrinmer, 
€q that future repetitions rould btemuiimiMed or avoided ..entirely • 



t future repetitions rould btemuij 
ied"below are several examplfls of 



'' Diseu?sed-below' are several examplfls of these .kinds of errors." , 

■%e.user asHed, for help '(A) , and received the list of titles for the * , 

• .eighfe-feapes^g^ TtelplihraTT. Thta^aa-JgAlowfd by the tnst ruction -to 

1 ^ - V e nEcr^ g he of tho inr^jf^ai-^ng a ehel ee from the Ilstf/ or' alss .to exit 

^ - tdlh.a DfMiOG Go^md/ Befo'jra user had aade l^^disiM, IIDA printsd 

^rtflie nessage that she had entered a DWLOG TOsnaand, atfid' therefore she had 



left Wmlp md retoimed ^rtte aa^ profras, A DIALOG pri^pt Allowed. 



elidt€# f rofff "frh^ Help J^fi"^ Because tte iisar did not mter ^ti^^ 

. - . ^\ . ^ ... ..... 

gpg^ndf ; the 1 ajlk ya^^ assi^ed f he . mtegea# m jiys tax error was regorted^ 
The lim staff re»rded rthlg mm a te^mieal^^errar ratter syntax 

#rror siJiM it obviously the fa,ult of the ^stem* ^ | 

^ ^ ^ ^ ^ - ^ ^ . ^- . ■ ^ 

Onm i^er canoeUed":^^ta(ro eharaaters^ and eohtinjied t© rfr*type thtf^^ : 



eoaieiand correctly. Nevertheless # one ©f the chaMieters ms picked \jp^ an^"^ ' ; V 



IIEA rejtoted- tiie c^Wand as a #yn tot er^»^ to©e users r V^^ entering 
the BfiGIK coimand^ s^h as BEOTi 8^ received a syntax error message- 



explaining that argmi^nt *8 ' was not acoeptable as csrily data bases 1# 6, 

- .... 1. ^ii^. . ..... :. . . 

S#_and_201-^ are„.all^jid iy HDA*- Other, users recMv^JCIQ^^syni 
mas sages r after oom^ining two^iegi^jnate sets* wnera'fihe oessage ciaSaett^ 
tha^ one of the sets^was null. All of thasa "arrors" wew re-^dfed as 
technical errors * / ■ ' * 

One legitittate syntax ef rdr ocllte when the user "jumps the gun" by 



\ 



entering the Initial part of a oopmand^ or the Abbreviation of a coniiiand|r 
bafpre $he ^IMM prqiapjt is is sued . this occurs # the user 5^y receive 

a ®ess*ge.. that' raa^^ "arfument does not begin with a nuobeir as it should." 

Ifi one casei the uset iade an additional arror because the original error 
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aessaga was misimderstood . Thi^jiuser re-*entered ^e caomand prefixed ^y 



a nmaber. and a serond sTOtaar error w»s reported^ The IIDA 'sta£f3is% 
ttmited tills seTOrsd error, since it was causiT by the mlsleadto^ IID^ 
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